Skip to content

Implement soft delay extension #264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 22 commits into
base: master
Choose a base branch
from

Conversation

btasdelen
Copy link
Collaborator

@btasdelen btasdelen commented Mar 10, 2025

Work in progress. Effort to implement soft delay extension for file format 1.5.0 compatibility.

TODO:

  • Add soft_delay handling to addBlock().
  • Implement apply_soft_delay().
  • Add soft_delay handling to write().
  • Add soft_delay handling to read().
  • Implement get_default_soft_delay_values()
  • Update check_timing().
  • Implement soft_delay handling in seq.plot().
  • Implement register_soft_delay_event()
  • Add tests for soft delay.
  • Add example sequence.

Challenges/Concerns:

  • Adding soft delays seemed to change some extension ID's, causing sequence tests to fail as the .seq files do not match. Invesigate/fix.
  • I don't think Pulseq handles soft delays in checkTiming() and seq.plot(). What should we do?

@btasdelen btasdelen mentioned this pull request Mar 1, 2025
11 tasks
@btasdelen btasdelen marked this pull request as draft March 10, 2025 03:38
Copy link

github-actions bot commented Mar 10, 2025

Coverage

Coverage Report
FileStmtsMissCoverMissing
/home/runner/.local/lib/python3.12/site-packages/pypulseq
   add_gradients.py1235159%44, 52, 58, 61, 75–86, 92, 120–123, 130–131, 150, 157, 162–241
   add_ramps.py36360%1–89
   align.py35489%41, 45, 69, 73
   calc_duration.py25196%37
   calc_ramp.py2182142%45–353
   calc_rf_bandwidth.py272026%37–59, 63–67
   check_timing.py962970%78, 82, 107, 180, 198, 231, 238, 248–292
   compress_shape.py30197%28
   convert.py40880%42, 48, 66, 72–73, 82, 88–89
   event_lib.py961485%6–9, 48–51, 70–71, 205–210
   make_adc.py921386%63, 72–76, 79, 128, 131, 135, 141, 145, 184, 186, 188, 196
   make_adiabatic_pulse.py1293970%196–200, 217–221, 229–230, 253, 259, 328–347, 451–460, 498–506
   make_arbitrary_grad.py37781%68, 71, 74, 77, 81, 83, 103
   make_arbitrary_rf.py665517%83–160
   make_block_pulse.py46393%112–116, 119
   make_delay.py9189%27
   make_digital_output_pulse.py16288%39, 47
   make_extended_trapezoid.py561279%67, 70, 76, 82, 85, 88, 91, 94, 116, 134, 136, 139
   make_extended_trapezoid_area.py93397%52, 227, 230
   make_gauss_pulse.py692071%127–131, 134–158, 165, 168
   make_label.py22482%64, 66, 68, 75
   make_sigpy_pulse.py1163173%12–13, 112, 115, 119, 154, 157–161, 165, 168–169, 172–173, 188, 195, 200, 212, 215, 240–250, 264, 267, 297–307
   make_sinc_pulse.py681085%94, 100, 127–131, 135, 138–139, 142–143, 165
   make_soft_delay.py17382%48, 51, 61
   make_trapezoid.py111794%177, 190, 196, 214, 232, 237, 255
   make_trigger.py16288%44, 52
   opts.py66986%78, 83, 102, 142, 166–170
   points_to_waveform.py9189%27
   rotate.py691480%15, 55, 66–69, 85–90, 112, 119–120
   scale_grad.py14471%28–30, 33
   sigpy_pulse_opts.py26773%34–41
   split_gradient.py393121%46–103
   split_gradient_at.py702761%63–90, 110, 114, 118–120, 154–156
   traj_to_grad.py13931%26–40
/home/runner/.local/lib/python3.12/site-packages/pypulseq/SAR
   SAR_calc.py1139813%33–40, 55–62, 89–108, 129–132, 168–212, 242–246, 264–306
/home/runner/.local/lib/python3.12/site-packages/pypulseq/Sequence
   block.py4013891%63, 66, 74, 80, 95, 103, 109, 120, 123, 126, 134, 139, 148, 159, 167, 208, 210, 214, 226, 275, 279, 295, 335–338, 367–368, 434, 440, 473, 542, 578, 584, 611, 649, 673, 712
   calc_grad_spectrum.py81766%68–190
   calc_pns.py403122%45–96
   ext_test_report.py1401192%23, 58, 61, 135, 227–233
   install.py754244%31, 52, 69, 71, 112–131, 148, 181–184, 200–212, 254–278
   parula.py4250%19–86
   read_seq.py3196879%42–43, 90, 93, 105, 110, 116, 123, 132, 141, 146, 149, 157–159, 202, 207, 215–264, 294–297, 312–313, 342–359, 422, 425, 460, 468, 542, 584–588
   sequence.py79527565%11–14, 104–114, 135–148, 186, 251–254, 301, 328, 345, 393, 421, 448–453, 490, 506, 597, 619, 660–663, 717, 755, 766–767, 773, 784, 790, 792, 800, 833–841, 862–884, 927, 929, 932, 958–959, 962–965, 1001–1011, 1020–1022, 1066–1078, 1093–1094, 1130–1131, 1157, 1163, 1166, 1169, 1206, 1327–1340, 1363, 1391, 1413–1415, 1436, 1472–1521, 1540–1584, 1611, 1622–1635, 1647–1658, 1704–1705, 1714–1732, 1756, 1786–1794, 1826–1936, 1972, 1986–1996, 2000, 2011
   write_seq.py35417650%42, 66, 69–76, 303–526
/home/runner/.local/lib/python3.12/site-packages/pypulseq/utils
   cumsum.py14193%17
   safe_pns_prediction.py12611310%50–87, 102–189, 197–214, 222, 244–250, 279–286, 310–336, 344–383, 396–411, 415
   tracing.py16662%33–34, 42, 54–55, 75
/home/runner/.local/lib/python3.12/site-packages/pypulseq/utils/siemens
   asc_to_hw.py58539%21–28, 48–106
   readasc.py48456%25–100
TOTAL4683172763% 

Tests Skipped Failures Errors Time
1295 20 💤 0 ❌ 0 🔥 3m 15s ⏱️

@FrankZijlstra
Copy link
Collaborator

Regarding the failing sequence tests, you can set the environment variable SAVE_EXPECTED=1 and run pytest to locally rewrite the .seq files. This will automatically pass the write tests, but then allows you to do a diff with the previous .seq files (e.g. git diff, or make a copy of the .seq files first). Then if you manually verify that the change of IDs is expected, commit the new .seq files. I know manually checking this is a bit of a pain, but I think aiming for a exact match of the .seq files is a good standard, better safe than sorry.

@btasdelen btasdelen marked this pull request as ready for review June 6, 2025 22:35
@btasdelen
Copy link
Collaborator Author

I think this is ready for testing, but there are several things that may require polishing.

  1. I did not add any tests because I could not come up with reasonable unit tests for this.
  2. I did not see specific changes to plot() or checkTiming() functions in Matlab version, so I couldn't figure out how/if they handle the plotting of soft delays.
  3. This required allowing float input to add_block() function to mimic the behavior in Matlab, which to my understanding is to mainly set required_duration variable. I feel like this could be a separate function parameter with a default value None. My only concern with that is this variable only seems to be used for soft delay anyways, so why is this not a member of soft delay object? How much to diverge from Matlab?
  4. Should we merge this to main, or a separate 1.5.0 branch? With the example scripts, without using any soft delays, there is no change to .seq files, so I hope it is safe to merge to main.

based on the user input (to the interpreter) according to the equation dur=input/factor+offset.
Required parameters are 'numeric ID' and 'string hint'. Optional parameter 'factor' can be either
positive and negative. Optional parameter 'offset' given in seconds can also be either positive and
negative. The 'hint' parameter is expected to be for identical 'numID'.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'hint' parameter is expected to be identical (?) for identical 'numID'.

from pypulseq.supported_labels_rf_use import get_supported_labels
from pypulseq.utils.cumsum import cumsum
from pypulseq.utils.tracing import format_trace, trace, trace_enabled

major, minor, revision = __version__.split('.')[:3]


class BiDict:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this really necessary?
do we need lookup by "id" and by "hint"==name?

cant we just assume that insertion order is equal to id?

or is there an actual use case where one wants to specify the soft delay id when creating it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems tedious to me to have both id and hint, which according to the documentation for make_soft_delay needs a one-to-one relation anyway. I don't see why we can't just refer to a delay always by name.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quoting myself from another feedback:

I think the main purpose of the numID is letting the user choose which delay box they want to use for a specific delay. For example, if they want to put TE first then TR, they can do so by explicitly assigning IDs 0 and 1, to them, regardless of the order they are added into the sequence.

But, I agree with you. It creates confusion for little gain. Also, I tried setting numID to 2 and did not assign anything to 0 and 1, and interpreter shows 3 delays anyway since it has to create them sequentially. It gracefully handled empty events, but still.

I think we can deviate from the Matlab here.

Copy link
Collaborator

@fzimmermann89 fzimmermann89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a quick look -- and noticed i need more time too understand the logic and play around with the code....

@fzimmermann89
Copy link
Collaborator

fzimmermann89 commented Jun 13, 2025

regarding check timings: for it to make sense, it would need to get the information about the values that should be used during the check, correct? maybe a dict of values?
and check_timings should warn that the timings are only checked for this particular setting of delay values, other settings in the scanner UI can break.

or would the expected workflow be save seq, apply_soft_delay for all delays, check_timings ?
then check_timings should complain if there are any "unapplied" soft delays. or am i misunderstanding how soft delays are supposed to work?

@fzimmermann89
Copy link
Collaborator

PS: I am stricly in favor of diverting from matlab to make parameters and functions more explicit and pythonic.

Comment on lines 111 to 112
10e-6, # Add a small delay as we can't have soft_delay in an 0 duration block.
pp.make_soft_delay(numID=0, hint='TE', offset=-min_TE, factor=1.0),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really dislike this as it is counter-intuitive (soft delay blocks must be empty, but then must have a duration...). I would prefer if make_soft_delay has a default delay value (required to be >0) that gets used as the block duration.

On a sidenote, I really think the float input as an option to add_block should not exist in the first place.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. For now, I removed the required_duration, now default_duration is an input to make_soft_delay with a default value of 10-e5.

On a sidenote, I really think the float input as an option to add_block should not exist in the first place.

I looked into this, but I felt like removing that will cause bunch of other issues to solve: 99bbffd


Parameters
----------
numID : int
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the reason the numID parameter exists and is not just assigned automatically, as is done for other events. If the hint needs to be identical for identical numID, then the delay can be uniquely addressed with just the hint.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the main purpose of the numID is letting the user choose which delay box they want to use for a specific delay. For example, if they want to put TE first then TR, they can do so by explicitly assigning IDs 0 and 1, to them, regardless of the order they are added into the sequence.

But, I agree with you. It creates confusion for little gain. Also, I tried setting numID to 2 and did not assign anything to 0 and 1, and interpreter shows 3 delays anyway since it has to create them sequentially. It gracefully handled empty events, but still.

I think we can deviate from the Matlab here.

if hint not in sd_str2numID:
raise ValueError(f"Specified soft delay '{hint}' does not exist in the sequence")

def get_default_soft_delay_values(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The functionality of get_default_soft_delay_values does not match its name. Checking consistency of the events should be a single function, and that function should be called somewhere to ensure the sequence is error-free. Or add the checks to check_timing (which is called by default from write_seq).

Getting default values should be another, separate function.

@FrankZijlstra
Copy link
Collaborator

General comment: It's probably a good idea to add write_gre_label_softdelay to the example sequence tests (test_sequence.py line 215).

@btasdelen
Copy link
Collaborator Author

@FrankZijlstra @fzimmermann89 Many thanks for the comments. It looks like #288 takes this PR and has many changes on top of this. I am not sure how to proceed with this without duplicating effort and creating some conflicts.

It looks like there are some changes to soft delay part, but most changes on #288 are for other features. If @mcencini wants to adress these feedback on there I can close this PR.

@mcencini
Copy link
Contributor

mcencini commented Jun 13, 2025

@btasdelen apologies, I did not mean to create confusion. The only changes in the soft-delay-part are some bug fixes and a simple test case that I wanted to add to this PR (see btasdelen#2). The idea was to develop the remaining v1.5.0 features mentioned in #258, but wait until this PR was merged.

That said, I am happy to help polishing this. If you think you don't need to have soft delay extension in 1.4.x, I can address it in #288. The only issue is that the v1.5.x changes are quite a lot and I unfortunately I do not see an easy way to split it in multiple smaller PR for easy review, except for soft delay which is quite independent from the rest. Suggestions are welcome, either in #288 or #258 threads! :D

@btasdelen
Copy link
Collaborator Author

@mcencini No worries. That is why I started with soft delays, it was pretty much independent of the rest of the stuff.

I can try cherry-picking your relevant commits. If I make changes and merge this, I believe you will have to deal with some conflicts when you update your branch.

I am fine either way. My opinion is, regardless who deals with the rest, it is better to divide things into smaller PRs, even if the individual PRs are not complete v1.5.0 compatible.

@btasdelen
Copy link
Collaborator Author

@fzimmermann89 @FrankZijlstra I had a crack at making numID optional, and making soft_delay_hints a normal dictionary to always refer soft delay's by their hints. I realized it is done to keep EventLibrary's data integer only. We don't have that restriction in Python, so I save the hint as string now. soft_delay_hints is a normal dict now, and its only purpose is to make sure numID -> hint mapping is unique.

Another way to approach this is to make use of the soft_delay_hints dictionary, and just not store hint in EventLibrary, instead we can refer to delays by their numID. For this, though, I believe I need to make soft_delay_hints a bijective dictionary again to fetch numID when user provided hint.

Also, I did not remove numID input but made it optional. Let me know if this looks reasonable.

@btasdelen
Copy link
Collaborator Author

@fzimmermann89 @FrankZijlstra I think this is the only thing left before the v1.5.0 release. Let me know if you have any further comments. I know it is not perfect, but one option to be just merge this and see if folks use it and have any bug reports on it. Since this is pretty much decoupled from the rest of the code, it should be okay.

@FrankZijlstra
Copy link
Collaborator

I think it would be okay to merge it with a few minor changes:

  • Add write_gre_label_softdelay to the example sequence tests in test_sequence. This checks basic things like writing and reading the sequence, which is good to test the soft delay extension blocks in the sequence file.
  • Fix get_default_soft_delay_values so it does not contain the error report stuff. I do not fully understand the checks being done, but it seems to me it should be in check_timing (or maybe it can be checked immediately when the block gets added?). These can be left as placeholders wherever it makes most sense to put these checks. Note that check_timing automatically gets called before writing a sequence, so it is the most safe for any sequence checks that would breaks things on the scanner.

@btasdelen
Copy link
Collaborator Author

@FrankZijlstra

  • Done.
  • I moved the remaining timing check into check_timing and removed the error_report stuff. I kept the other checks as they are not timing checks, but I can remove them if you want to. I can even remove the whole get_default_soft_delay_values function if it bothers you, because I don't really understand its purpose or if it works properly...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants