feature: IMDReader Integration #4923

amruthesht · 2025-02-19T22:47:49Z

This draft PR addresses the feature request discussed in #4827.

Note:
The IMDReader feature which was previously a part of the imdclient package has been moved into MDAnalysis below. Any other modules have been in retained in imdclient, which has been added as an optional dependency here. We are currently in the process of splitting the imdclient package as mentioned above. (Issue, PR)

Major changes made in this Pull Request:

IMDReader, other associated base classes and a utility function were added to coordinates in the main package.
Appropriate test cases for the reader were added as a part of test-imd.py
Other corresponding changes were made to CI and environment *.yaml files
imdclient was added as an optional dependency

PR Checklist

Issue raised/referenced?
Tests updated/added?
Documentation updated/added?
package/CHANGELOG file updated?
Is your name in package/AUTHORS? (If it is not, add it!)

Developers Certificate of Origin

I certify that I can submit this code contribution as described in the Developer Certificate of Origin, under the MDAnalysis LICENSE.

📚 Documentation preview 📚: https://mdanalysis--4923.org.readthedocs.build/en/4923/

github-actions

Hello there first time contributor! Welcome to the MDAnalysis community! We ask that all contributors abide by our Code of Conduct and that first time contributors introduce themselves on GitHub Discussions so we can get to know you. You can learn more about participating here. Please also add yourself to package/AUTHORS as part of this PR.

codecov · 2025-02-19T22:58:47Z

Codecov Report

❌ Patch coverage is 96.53179% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.88%. Comparing base (5d48c5c) to head (ea85bb5).
⚠️ Report is 1 commits behind head on develop.

Files with missing lines	Patch %	Lines
package/MDAnalysis/coordinates/IMD.py	93.02%	0 Missing and 6 partials ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #4923      +/-   ##
===========================================
+ Coverage    93.86%   93.88%   +0.02%     
===========================================
  Files          179      180       +1     
  Lines        22249    22422     +173     
  Branches      3161     3186      +25     
===========================================
+ Hits         20885    21052     +167     
  Misses         902      902              
- Partials       462      468       +6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

amruthesht · 2025-02-19T23:02:30Z

Your thoughts on this are appreciated - @orbeckst

orbeckst

Thanks for the initial PR.

The first big step is to get the tests running properly so that the CI uses an imdclient without IMDReader. Otherwise we are not sure we're testing the code here.
Minor initial comments while I skimmed.
Simple thing: run black over all files to get the formatting and ordering of imports right

.github/actions/setup-deps/action.yaml

package/MDAnalysis/coordinates/IMD.py

package/MDAnalysis/coordinates/base.py

package/MDAnalysis/coordinates/util.py

package/pyproject.toml

testsuite/MDAnalysisTests/coordinates/test_imd.py

package/MDAnalysis/coordinates/IMD.py

orbeckst · 2025-02-19T23:50:31Z

I set the PR to Work in progress for the time being, just to indicate that we're not yet at the stage where the CI is working. Once the tests run properly, we can update the status.

Obviously, this shouldn't discourage anyone from contributing and commenting.

IAlibay

A first quick look

testsuite/MDAnalysisTests/coordinates/test_imd.py

jaclark5

I read through and nothing jumped out at me that wasn't already mentioned, except that I didn't see these changes in the documentation that was linked. The following needs to be added:

Add doc/sphinx/source/documentation_pages/coordinates/IMD.rst
.. automodule:: MDAnalysis.coordinates.IMD

doc/sphinx/source/documentation_pages/coordinate_modules.rst
coordinates/IMD

doc/sphinx/source/documentation_pages/references.rst
If you use IMD capability...

hmacdope

Sorry for the delay, had a lookover, I will try and push some changes addressing some of these myself also, but would be good to pick up the momentum here again if possible.

package/MDAnalysis/coordinates/IMD.py

package/MDAnalysis/coordinates/base.py

testsuite/MDAnalysisTests/coordinates/test_imd.py

hmacdope · 2025-03-27T09:55:18Z

@amruthesht do we have a plan for pushing ahead with suggested changes?

amruthesht · 2025-03-27T15:33:14Z

@amruthesht do we have a plan for pushing ahead with suggested changes?

Working on the imdclient repository right now. I had a few small issues there with using the IMDReader, but getting them fixed soon.
And then I'll wrap up some small changes on the mdanalysis end and we should be good to go!

1. Moved `parse_host_port` to `IMD.py` and deleted `util.py` 2. Cleaned up `test_imd.py` - changes to `assert_*` functions and simplified non-applicable test to pass automatically

hmacdope · 2025-05-05T15:11:49Z

@amruthesht kicking CI here.

hmacdope · 2025-05-05T15:12:04Z

@yuxuanzhuang would you be able to have a run through?

yuxuanzhuang · 2025-05-05T16:04:55Z

I will have another look at the PR this week. One immediate thing is to add documentation.

Please refer to the comments below.

I read through and nothing jumped out at me that wasn't already mentioned, except that I didn't see these changes in the documentation that was linked. The following needs to be added:

Add doc/sphinx/source/documentation_pages/coordinates/IMD.rst .. automodule:: MDAnalysis.coordinates.IMD

doc/sphinx/source/documentation_pages/coordinate_modules.rst coordinates/IMD

doc/sphinx/source/documentation_pages/references.rst If you use IMD capability...

hmacdope · 2025-05-05T19:33:43Z

Need to figure out tests also @amruthesht

package/MDAnalysis/coordinates/timestep.pyx

1. API doc on multiple connections 2. Jupyter usage of `close()` for secure closure of connections

Replace `IMDModuleStateManager` with standard pytest patterns

hmacdope · 2025-09-08T05:30:36Z

@amruthesht I think cd21d6bf4892bc8172f905816beffc1b6fff8369 has messed up the import guard somehow. I would consider reverting possibly.

This reverts commit cd21d6b.

orbeckst · 2025-09-08T17:21:24Z

@hmacdope @yuxuanzhuang what needs to be done to get the PR merged from your end?

(I had a cursory look at @IAlibay 's #4923 (review) and it seems to have been addressed but it's ultimately up to @IAlibay .)

- move details on which engines support multiple connections to imdclient docs (only update one place) and link - rearranged warning/seealso boxes

- state for each engine if they support concurrent connections - minor markup/link reformatting - moved information from MDAnalysis/mdanalysis#4923 to here

hmacdope · 2025-09-08T23:07:11Z

Quick weigh in from @IAlibay either way and I think we are good to go @orbeckst

IAlibay

Sorry for the delay, pings didn't show up in my github feed.

Couple of cosmetic things and then this is good from me.

.github/actions/setup-deps/action.yaml

package/CHANGELOG

IAlibay · 2025-09-18T17:23:21Z

package/doc/sphinx/source/documentation_pages/coordinates_modules.rst

   coordinates/GSD
   coordinates/GRO
   coordinates/H5MD
+   coordinates/IMD


Remember to open up a PR in the userguide to update the tables.

Updated current Issue #427 with a note about it. I'll work on a PR to make necessary updates to the UserGuide based on it.

package/pyproject.toml

orbeckst · 2025-09-19T16:49:10Z

Once this is merged, we (= @amruthesht @HeydenLab @orbeckst ) would like to have a rc on pypi so that we can have at least one installable package for the workshop. We also need to update the codespace/docker images so the sooner we can have it, the better.

Co-authored-by: Irfan Alibay <[email protected]>

IAlibay · 2025-09-19T19:36:49Z

rc on pypi

@orbeckst RC of MDAnalysis?

orbeckst · 2025-09-19T19:40:27Z

Yes – a 2.10.0-rc1 pypi package so that we can have a "normal" pip install everywhere and when we get 2.10.0 out just before the UGM we have a simple update to the installation instructions (EDIT: and all the scripts inside the containers for the workshop that is supposed to run in codespace).

IAlibay · 2025-09-19T20:28:59Z

Yes – a 2.10.0-rc1 pypi package so that we can have a "normal" pip install everywhere and when we get 2.10.0 out just before the UGM we have a simple update to the installation instructions (EDIT: and all the scripts inside the containers for the workshop that is supposed to run in codespace).

How about just cutting a new release? At that point it would be the same amount of work I think. -- edit: I misread what you meant, sorry! I'm not super sure about doing a pypi rc, it's doable for a tarball but we won't be able to do wheels.

Maybe TBD in a call at some point?

1. Alphabetical order for `imclcient` as optional dependency 2. Changed order in `CHANGELOG`

amruthesht · 2025-09-19T22:28:44Z

@orbeckst - All remaining requested changes are done. Please let me know if there is anything else

hmacdope · 2025-09-23T09:36:17Z

Congratulations @MDAnalysis/streaming!

orbeckst · 2025-09-23T16:25:58Z

@IAlibay ,

How about just cutting a new release? At that point it would be the same amount of work I think. -- edit: I misread what you meant, sorry! I'm not super sure about doing a pypi rc, it's doable for a tarball but we won't be able to do wheels.

Maybe TBD in a call at some point?

A proper 2.10.0 would also work, of course.

I assume a tarball would also work but perhaps we'll move the initial discussion to discord.

orbeckst · 2025-09-23T16:26:36Z

And: hooray 🎉 !

Thank you everyone for all your hard work.

IAlibay · 2025-09-23T16:34:13Z

I assume a tarball would also work but perhaps we'll move the initial discussion to discord.

Yes, let's do that (move to discord), we're overdue a new release I think, python 3.14 is out in two-ish weeks.

amruthesht added 2 commits February 19, 2025 08:23

IMDReader integration

a6f2bb2

small cleanup

a67cbfb

github-actions bot reviewed Feb 19, 2025

View reviewed changes

orbeckst requested changes Feb 19, 2025

View reviewed changes

IAlibay reviewed Feb 19, 2025

View reviewed changes

package/MDAnalysis/coordinates/IMD.py Show resolved Hide resolved

orbeckst marked this pull request as draft February 19, 2025 23:48

orbeckst assigned hmacdope and yuxuanzhuang Feb 19, 2025

IAlibay requested changes Feb 20, 2025

View reviewed changes

jaclark5 requested changes Feb 23, 2025

View reviewed changes

hmacdope requested changes Mar 23, 2025

View reviewed changes

amruthesht and others added 2 commits April 9, 2025 17:12

Chore: Cleanup and small fixes

8c00bc1

1. Moved `parse_host_port` to `IMD.py` and deleted `util.py` 2. Cleaned up `test_imd.py` - changes to `assert_*` functions and simplified non-applicable test to pass automatically

bump CI

bc88b7b

hmacdope requested a review from yuxuanzhuang May 5, 2025 15:12

Add minimum docs

cf15cf9

jaclark5 self-requested a review May 24, 2025 23:52

hmacdope added 5 commits June 7, 2025 15:00

Merge remote-tracking branch 'upstream/develop' into imdv3-dev

7aed3b4

try add mock for import versions

b5ff03d

fix for wrong assert

6263151

fix fixture issues

efbb903

add short class description

2ff3935

Modify: timestep.pyx to remove usage of new keyword

16454a6

amruthesht commented Aug 28, 2025

View reviewed changes

package/MDAnalysis/coordinates/timestep.pyx Show resolved Hide resolved

amruthesht commented Aug 28, 2025

View reviewed changes

package/MDAnalysis/coordinates/timestep.pyx Show resolved Hide resolved

amruthesht requested a review from IAlibay August 28, 2025 18:04

amruthesht added 4 commits September 2, 2025 13:54

Docs: Updated

19a2526

1. API doc on multiple connections 2. Jupyter usage of `close()` for secure closure of connections

Cleanup: Remove timestep comment

b465bf1

Bug Fix: Warnign message for incompatible version

5138b54

Refactor: simplify IMD tests with pytest monkeypatch

cd21d6b

Replace `IMDModuleStateManager` with standard pytest patterns

Revert "Refactor: simplify IMD tests with pytest monkeypatch"

feb6c9d

This reverts commit cd21d6b.

IMDReader docs: reorganized and reduced multiple connection details

35fcc8e

- move details on which engines support multiple connections to imdclient docs (only update one place) and link - rearranged warning/seealso boxes

orbeckst mentioned this pull request Sep 8, 2025

Update IMDv3 support details for GROMACS, NAMD, and LAMMPS Becksteinlab/imdclient#114

Open

4 tasks

IAlibay requested changes Sep 18, 2025

View reviewed changes

Update package/pyproject.toml

f2a5050

Co-authored-by: Irfan Alibay <[email protected]>

Chore: Minor cleanup

ea85bb5

1. Alphabetical order for `imclcient` as optional dependency 2. Changed order in `CHANGELOG`

amruthesht requested a review from IAlibay September 19, 2025 22:27

IAlibay approved these changes Sep 22, 2025

View reviewed changes

hmacdope merged commit 519ac56 into MDAnalysis:develop Sep 23, 2025
23 checks passed

feature: IMDReader Integration #4923

feature: IMDReader Integration #4923

Uh oh!

Conversation

amruthesht commented Feb 19, 2025 • edited by orbeckst Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Checklist

Developers Certificate of Origin

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Feb 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

amruthesht commented Feb 19, 2025

Uh oh!

orbeckst left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

orbeckst commented Feb 19, 2025

Uh oh!

IAlibay left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jaclark5 left a comment

Choose a reason for hiding this comment

Uh oh!

hmacdope left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hmacdope commented Mar 27, 2025

Uh oh!

amruthesht commented Mar 27, 2025

Uh oh!

hmacdope commented May 5, 2025

Uh oh!

hmacdope commented May 5, 2025

Uh oh!

yuxuanzhuang commented May 5, 2025

Uh oh!

hmacdope commented May 5, 2025

Uh oh!

Uh oh!

Uh oh!

hmacdope commented Sep 8, 2025

Uh oh!

orbeckst commented Sep 8, 2025

Uh oh!

hmacdope commented Sep 8, 2025

Uh oh!

IAlibay left a comment

Choose a reason for hiding this comment

Uh oh!

amruthesht commented Feb 19, 2025 •

edited by orbeckst

Loading

codecov bot commented Feb 19, 2025 •

edited

Loading

orbeckst left a comment •

edited

Loading

orbeckst commented Sep 19, 2025 •

edited

Loading

IAlibay commented Sep 19, 2025 •

edited

Loading

IAlibay commented Sep 23, 2025 •

edited

Loading