Add new actions and improves handling of stochastic actions #449

Shaobo-Zhou · 2025-09-06T13:27:04Z

Description

This PR introduces new actions to the reinforcement learning environment and adds support for stochastic mapping passes:

🚀 Major Changes
Expanded Action Space

Added AIRouting as a routing/mapping option and wrapped in SafeAIRouting for stable integration into the RL pipeline.
Added tket layout actions: GraphPlacement and NoiseAwarePlacement (fidelity-aware).

Support for Stochastic Passes

Wrapped stochastic actions (e.g., AIRouting, SabreLayout) in a multi-trial evaluation loop, optimizing for the figure of merit instead of the default gate count internally in Qiskit passes.
Introduced max_iterations as parameters to control trial counts, enabling improved predictor performance and stability.

**Changes in determine_valid_actions_for_state**

Refined the logic: After a circuit is mapped, only mapping-preserving optimization actions are allowed to prevent reusing outdated layouts when the circuit structure changes.

Fixes and Enhancements

Fixed a bug in OptimizeCliffords by ensuring CollectCliffords runs beforehand.
Corrected computation of estimated success probability in reward.py
Fixed incorrect usage of GatesInBasis in rl/predictorenv.py
Adjusted benchmark level to INDEP in test_predictor_rl.py, since the current action space does not guarantee support for high-level gates.
Disabled multiprocessing on Windows with Python 3.13 due to _posixsubprocess issues.

Dependency Update

Added qiskit-ibm-ai-local-transpiler to the dependencies
Pinned networkx==2.8.5 to ensure compatibility with qiskit-ibm-ai-local-transpiler
Upgraded pytket_qiskit>=0.71.0

Checklist:

The pull request only contains commits that are focused and relevant to this change.
I have added appropriate tests that cover the new/changed functionality.
I have updated the documentation to reflect these changes.
I have added entries to the changelog for any noteworthy additions, changes, fixes, or removals.
I have added migration instructions to the upgrade guide (if needed).
The changes follow the project's style guidelines and introduce no new warnings.
The changes are fully tested and pass the CI checks.
I have reviewed my own code changes.

Update action space and feature space Update actions Update action space

Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations Remove example_test.py Remove example_test.py

Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations

Signed-off-by: Shaobo-Zhou <[email protected]>

Fix bugs Fix bugs Fix bugs

…e gate check

Fix windows runtime warning problem Fix windows runtime warning issue

denialhaag · 2025-09-25T10:20:51Z

Hey @Shaobo-Zhou, we just merged #468 to fix the CI on main. The PR contains the minimum version bumps necessary to make everything work again. I hope the rebase is not too bad. 🙂

noxfile.py

pyproject.toml

denialhaag · 2025-09-25T11:12:25Z

This seems to have worked fine! I just had two quick points regarding the rebase. 🙂

Signed-off-by: Shaobo-Zhou <[email protected]>

Shaobo Zhou and others added 30 commits March 29, 2025 19:20

Update predictor(adding callbacks)

129b60f

Update

08889bd

Restore helper.py and predictor.py to match upstream

e2ff3fe

Merge remote-tracking branch 'upstream/main'

1c32d15

Implement new mapping actions

78dc1aa

Update action space and feature space Update actions Update action space

Fix: resolve pre-commit issues and add missing annotations

a3ba836

Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations Remove example_test.py Remove example_test.py

Fix: resolve pre-commit issues and add missing annotations

5935e6f

Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations

Fix: resolve pre-commit issues and add missing annotations

f71fb29

Fix: resolve pre-commit issues and add missing annotations

3c7592b

Fix mypy errors

6db5c27

Fix mypy errors

47841c5

Fix dependencies issues

b1ac8ce

Fix dependency issues

5f8473c

Add missing zip file

7491ec0

Fix issue with Python 3.13

3346842

Merge branch 'main' into hybrid-mapping

6f7a73c

Signed-off-by: Shaobo-Zhou <[email protected]>

Remove Python 3.13 from noxfile.py due to compatibility issue

6c67349

Skip minimums session on Windows due to CI slowness

2692b96

Fix bugs

f4874e6

Fix bugs Fix bugs Fix bugs

Fix bugs

54eec91

Use default Qiskit settings for VF2Layout and add assertion for nativ…

845f7de

…e gate check

Debug

3418936

Fix missing argument

ae870cc

Fix warning issues

861bc62

Fix window runtime warning problem

fa989b6

Fix window runtime warning problem

405bd39

Add time limit for VF2PostLayout

7b2f321

Fix windows runtime warning problem

b67d0a6

Fix windows runtime warning problem Fix windows runtime warning issue

Add new actions

bf7c9ee

Add new actions

6d2733f

Shaobo-Zhou force-pushed the new_structure branch 3 times, most recently from 7644681 to 44e0e40 Compare September 23, 2025 17:31

Clean up venv after each session to free up space

350bae5

Shaobo-Zhou force-pushed the new_structure branch from 985671d to 350bae5 Compare September 23, 2025 18:40

Clean up venv after each session to free up space

91208d1

Shaobo-Zhou force-pushed the new_structure branch from 154b25c to 91208d1 Compare September 23, 2025 20:46

Clean up venv after each session to free up space

622d409

Shaobo-Zhou force-pushed the new_structure branch from 7f5f521 to 622d409 Compare September 23, 2025 23:40

denialhaag requested changes Sep 25, 2025

View reviewed changes

noxfile.py Outdated Show resolved Hide resolved

pyproject.toml Show resolved Hide resolved

Shaobo-Zhou force-pushed the new_structure branch 4 times, most recently from 5ccc3c0 to b28ccf2 Compare September 25, 2025 17:44

Update noxfile

fa008a6

Shaobo-Zhou force-pushed the new_structure branch from 5614a9f to fa008a6 Compare September 25, 2025 18:11

Update ibm runtime dependency

0c98e56

Shaobo-Zhou force-pushed the new_structure branch from 48c7aa8 to 0c98e56 Compare September 25, 2025 19:13

Update noxfile

34a1c7c

Shaobo-Zhou force-pushed the new_structure branch from 268e947 to 34a1c7c Compare September 25, 2025 21:05

Update noxfile

a8d069a

Shaobo-Zhou force-pushed the new_structure branch from 99c7847 to a8d069a Compare September 25, 2025 22:25

Update noxfile

114b79b

Shaobo-Zhou force-pushed the new_structure branch from 8987453 to 114b79b Compare September 25, 2025 23:57

Fetch update from main

aaf14b1

Shaobo-Zhou force-pushed the new_structure branch from 07ec60f to aaf14b1 Compare September 26, 2025 13:45

Shaobo-Zhou and others added 2 commits September 26, 2025 15:48

Merge branch 'main' into new_structure

46da095

Signed-off-by: Shaobo-Zhou <[email protected]>

🎨 pre-commit fixes

97af7bb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add new actions and improves handling of stochastic actions #449

Add new actions and improves handling of stochastic actions #449

Uh oh!

Shaobo-Zhou commented Sep 6, 2025 •

edited

Loading

Uh oh!

denialhaag commented Sep 25, 2025

Uh oh!

Uh oh!

Uh oh!

denialhaag commented Sep 25, 2025

Uh oh!

Uh oh!

Uh oh!

Add new actions and improves handling of stochastic actions #449

Are you sure you want to change the base?

Add new actions and improves handling of stochastic actions #449

Uh oh!

Conversation

Shaobo-Zhou commented Sep 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist:

Uh oh!

denialhaag commented Sep 25, 2025

Uh oh!

Uh oh!

Uh oh!

denialhaag commented Sep 25, 2025

Uh oh!

Uh oh!

Shaobo-Zhou commented Sep 6, 2025 •

edited

Loading