Reprex for hanging pipeline using error="trim" with crew

[koefoeden]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

Hi! As requested, I managed to create a reprex for #1329 and #1340.

Can you check whether it reproduces for you?

library(targets)

tar_option_set(
    error = "trim",
    controller =  crew::crew_controller_local(name = "controller_A"),
    resources = tar_resources(crew = tar_resources_crew(controller = "controller_A"))
)

list(
    tar_target(
        erroring_target,
        stop()
    ),
    tar_target(
        downstream_target_1,
        {
            erroring_target
            Sys.sleep(5)
        }
    ),
    tar_target(
        downstream_target_2,
        {
            downstream_target_1
            Sys.sleep(5)
        }
    ),
    tar_target(
        downstream_target_3,
        {
            downstream_target_2
            Sys.sleep(5)
        }
    )
)

Only running the third downstream target makes it hang.

=> tar_make(matches("downstream_target_3"), script="trim_reprex.R")
+ erroring_target dispatched
✖ erroring_target errored  
**hanging**

While curiously, the others seem to work (i.e. correctly conclude with an error)

=> tar_make(matches("downstream_target_2"), script="trim_reprex.R")
+ erroring_target dispatched
✖ erroring_target errored                                  
✖ errored pipeline [4.5s, 0 completed, 0 skipped] 

It might be possible to trim it down further - but this is a good starting point I think!

Session Info

> sessioninfo::session_info()                                                                                                                                                                                                               
─ Session info ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
 setting  value
 version  R version 4.3.3 (2024-02-29)
 os       Red Hat Enterprise Linux 8.10 (Ootpa)
 system   x86_64, linux-gnu
 ui       X11
 language (EN)
 collate  en_US.UTF-8
 ctype    en_US.UTF-8
 tz       Europe/Copenhagen
 date     2025-04-23
 pandoc   2.0.6 @ /usr/bin/pandoc
 quarto   NA

─ Packages ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
 ! package     * version date (UTC) lib source
 P backports     1.5.0   2024-05-23 [?] CRAN (R 4.3.3)
 P base64url     1.4     2018-05-14 [?] CRAN (R 4.3.1)
 P callr         3.7.6   2024-03-25 [?] CRAN (R 4.3.3)
 P cli           3.6.3   2024-06-21 [?] CRAN (R 4.3.3)
   codetools     0.2-19  2023-02-01 [2] CRAN (R 4.3.3)
 P data.table    1.16.0  2024-08-27 [?] CRAN (R 4.3.3)
 P fansi         1.0.6   2023-12-08 [?] CRAN (R 4.3.3)
 P glue          1.8.0   2024-09-30 [?] CRAN (R 4.3.3)
 P igraph        2.0.3   2024-03-13 [?] CRAN (R 4.3.3)
 P knitr         1.48    2024-07-07 [?] CRAN (R 4.3.3)
 P lifecycle     1.0.4   2023-11-07 [?] CRAN (R 4.3.3)
 P magrittr      2.0.3   2022-03-30 [?] CRAN (R 4.3.1)
 P pillar        1.9.0   2023-03-22 [?] CRAN (R 4.3.1)
 P pkgconfig     2.0.3   2019-09-22 [?] CRAN (R 4.3.1)
 P prettyunits   1.2.0   2023-09-24 [?] CRAN (R 4.3.1)
 P processx      3.8.4   2024-03-16 [?] CRAN (R 4.3.3)
 P ps            1.8.0   2024-09-12 [?] CRAN (R 4.3.3)
 P R6            2.5.1   2021-08-19 [?] CRAN (R 4.3.1)
 P renv          1.1.4   2025-03-20 [?] RSPM
 P rlang         1.1.4   2024-06-04 [?] CRAN (R 4.3.3)
 P rstudioapi    0.16.0  2024-03-24 [?] CRAN (R 4.3.3)
 P secretbase    1.0.3   2024-10-02 [?] CRAN (R 4.3.3)
 P sessioninfo   1.2.3   2025-02-05 [?] RSPM
 P targets     * 1.11.1  2025-04-10 [?] CRAN (R 4.3.3)
 P tibble        3.2.1   2023-03-20 [?] CRAN (R 4.3.1)
 P tidyselect    1.2.1   2024-03-11 [?] CRAN (R 4.3.3)
 P utf8          1.2.4   2023-10-22 [?] CRAN (R 4.3.3)
 P vctrs         0.6.5   2023-12-01 [?] CRAN (R 4.3.1)
 P withr         3.0.1   2024-07-31 [?] CRAN (R 4.3.3)
 P xfun          0.48    2024-10-03 [?] CRAN (R 4.3.3)
 P yaml          2.3.10  2024-07-26 [?] CRAN (R 4.3.3)

 [1] /maps/projects/cbmr_shared/people/tqb695/non-GDPR/reprex/renv/library/R-4.3/x86_64-pc-linux-gnu
 [2] /maps/direct/software/R/4.3.3/lib64/R/library

 * ── Packages attached to the search path.
 P ── Loaded and on-disk path mismatch.

[wlandau]

Thanks for the reprex! Now that I can reproduce the hanging on my end, I can step through a debugger to figure out what's going on, and now it's clear that the pipeline forgets to call targets:::target_update_queue() when it processes a trimmed target. Only a long chain of downstream targets could have possibly reproduced this. I never would have known.

I have a couple test cases:

library(targets)
tar_option_set(
  error = "trim",
  controller =  crew::crew_controller_sequential()
)
list(
  tar_target(erroring_target, stop()),
  tar_target(downstream_1, erroring_target),
  tar_target(downstream_2, downstream_1),
  tar_target(downstream_3, downstream_2),
  tar_target(dynamic, downstream_3, pattern = map(downstream_3)),
  tar_target(aggregate, dynamic)
)

library(targets)
tar_option_set(
  error = "trim",
  controller =  crew::crew_controller_sequential()
)
list(
  tar_target(a, seq_len(2)),
  tar_target(erroring_target, stop(), pattern = map(a)),
  tar_target(downstream_1, erroring_target, pattern = map(erroring_target)),
  tar_target(downstream_2, downstream_1, pattern = map(downstream_1)),
  tar_target(downstream_3, downstream_2),
  tar_target(dynamic, downstream_3, pattern = map(downstream_3)),
  tar_target(aggregate, dynamic)
)

Both fail with the current version of targets, and they both succeed if I insert tar_update_queue(target, schedule) in this conditional block. I'll push a patch soon.

[wlandau]

Fixed in dfa6eb9

[koefoeden]

Amazing! 🎉

[koefoeden]

Confirmed fixed in my real pipeline as well!