-
Notifications
You must be signed in to change notification settings - Fork 716
Open
Labels
Description
Bug report
I've been struggling with the startTask directive for Azure pools. Specifically, I've been having trouble using the startTask directive in combination with the default copyToolInstallMode. What seems to be happening is that this line in the nf-azure plugin is simply concatenating the startTask strings with a semicolon, but Azure is ignoring everything after the first semicolon.
Here's what's the pool startTask looks like on Azure with the default behavior (copyToolInstallMode = node):

And with the workaround (copyToolInstallMode = task):

Expected behavior and actual behavior
I'd expect commands passed via startTask directive to be executed.
Steps to reproduce the problem
Here's a minimal example showing the problem: https://github.com/tanaes/azure_starttask_test
Program output
(base) jonsan@nf-head:~/software/ramdisk_test$ nextflow run main.nf --storage_name $AZURE_STORAGE_ACCOUNT --storage_key $AZURE_STORAGE_ACCOUNT_KEY --batch_name $AZURE_BATCH_ACCOUNT
--batch_key $AZURE_BATCH_ACCOUNT_KEY -w "az://nextflow-data/projects/test/ramdisk_test/work"
Nextflow 25.06.0-edge is available - Please consider updating your version to it
N E X T F L O W ~ version 24.11.0-edge
Launching `main.nf` [trusting_lamarr] DSL2 - revision: 1aad912a88
executor > azurebatch (1)
[cf/283d62] process > TEST_STARTTASK [100%] 1 of 1 ✔
Testing if startTask was executed...
FAILURE: startTask not executed - file missing
Completed at: 22-Jul-2025 16:30:50
Duration : 3m 10s
CPU hours : (a few seconds)
Succeeded : 1
(base) jonsan@nf-head:~/software/ramdisk_test$ nextflow run main.nf --storage_name $AZURE_STORAGE_ACCOUNT --storage_key $AZURE_STORAGE_ACCOUNT_KEY --batch_name $AZURE_BATCH_ACCOUNT
--batch_key $AZURE_BATCH_ACCOUNT_KEY -w "az://nextflow-data/projects/test/ramdisk_test/work" --copyToolInstallMode task
Nextflow 25.06.0-edge is available - Please consider updating your version to it
N E X T F L O W ~ version 24.11.0-edge
Launching `main.nf` [magical_wegener] DSL2 - revision: 1aad912a88
executor > azurebatch (1)
[df/256182] process > TEST_STARTTASK [100%] 1 of 1 ✔
Testing if startTask was executed...
SUCCESS: startTask executed - file exists
Completed at: 22-Jul-2025 16:37:02
Duration : 3m 20s
CPU hours : (a few seconds)
Succeeded : 1
Environment
- Nextflow version: 24.11.0-edge
- Java version: openjdk version "19.0.2" 2023-01-17
- Operating system: Linux
- Bash version: 5.1.16(1)-release (x86_64-pc-linux-gnu)
Additional context
(Add any other context about the problem here)