Extended Parmest Capability for weighted SSE objective

[slilonfe5]

Fixes # .

Summary/Motivation:

Currently, the Parmest SSE objective does not support measurements in different units. This work adds new capabilities to Parmest, including weighted SSE to handle measurements in different units, and more robust covariance matrix calculation methods for more accurate uncertainty quantification. This work also enables the calculation of the covariance matrix using a user-supplied measurement error standard deviation.

Changes proposed in this PR:

• Added a weighted SSE objective
• Added two covariance matrix calculation methods for both the SSE and weighted SSE objectives
• Enabled calculation of the covariance matrix using a user-supplied measurement error standard deviation
• Added a separate function for the covariance matrix estimation

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

1. I agree my contributions are submitted under the BSD license.
2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

[slilonfe5]

@adowling2 @djlaky

[adowling2]

@slilonfe5 Here is some quick feedback

compute_jacobian function

• Make this a private method by adding _ to the function name
• Add as an argument to the function relative_perturbation
• In the document string, explain this is using forward (?) finite difference
• Add as an argument the solver object. You can make the default Ipopt.

Feedback on the compute_FIM method:

• Add relative_tolerance and solver as arguments
• Also add a check that error_list must be the same length as y_hat_list
• Add a debugging step for the linear algebra error, compute the condition number of the Jacobian matrix and print it out
• Why would you ever get a linear algebra error for just matrix multiplication? Is this check even needed?

[slilonfe5]

@adowling2 @djlaky I also updated the calculation for the normal SSE such that we can use the user-supplied measurement error if defined; otherwise, we calculate the measurement error as usual.

[adowling2]

Nice progress. I think it is time to start writing tests for the new capabilities.

[adowling2]

@slilonfe5 Once you have the tests ready, tag us for feedback. Also, I think you can skip adding this to the depreciated class.

[adowling2]

Here is more feedback as you work on getting this ready for the Pyomo team to review.

[slilonfe5]

@adowling2 @djlaky I have created a separate method (cov_est) for computing the covariance matrix, supporting three calculation methods (jacobian, kaug, and reduced_hessian). I implemented these covariance calculation methods for both the SSE and SSE_weighted objectives. Lastly, as you suggested, I did not add the new capability to the deprecated interface.

I tested these with three examples (2 steady state and 1 dynamic), and all work well. I'm yet to write the test file for these.

[mrmundt]

This seems like a lot of comments, but there are a few repeats! This is looking pretty good so far; most of my questions/suggestions aren't "necessary" but will make the code cleaner.

[slilonfe5]

@mrmundt Thank you for your feedback. I will work on them to make the code cleaner.

[slilonfe5]

@mrmundt @blnicho @jsiirola @adowling2 @djlaky I have implemented Miranda's final review on this PR. I will work on getting this moved to "Done" in the Parmest & Pyomo.DoE Development board. Thanks

[blnicho]

@slilonfe5 I took a look at the failing doctests and I think you just need to update some code snippets in the parmest documentation to match the API changes in your PR. This is the file and one of the doctests that's failing: https://github.com/Pyomo/pyomo/blame/10d6acc37cca0cf8bc1a5a9734516fc7fd70c989/doc/OnlineDocs/explanation/analysis/parmest/driver.rst#L86

Take a look at all the code snippets in that file and make sure they are using the updated API and won't throw deprecation warnings.

[slilonfe5]

@blnicho @mrmundt @jsiirola @adowling2 @djlaky I have implemented the feedback from Bethany regarding updating the documentation files (e.g., driver.rst) to match the API changes in this PR. All the documentation tests now pass. I need a final review of the updates I made to the documentation files; driver.rst, datarec.rst, and covariance.rst. Thank you.