From e53861bbb9056311bee0ae5de1c496e414171942 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Fri, 24 Oct 2025 14:45:19 +0200 Subject: [PATCH 01/17] Update Hello Nextflow training to use new output syntax (Nextflow 25.10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This commit modernizes the Hello Nextflow training materials to teach the new `publish:` and `output {}` syntax introduced in Nextflow 24.04+ and finalized in 25.10.0, while maintaining context about the legacy `publishDir` directive. ## Changes ### Documentation (6 files) - Updated all English lesson files to teach workflow outputs as primary approach - Added new section explaining workflow outputs vs legacy publishDir - Updated all version strings from 25.04.3 to 25.10.0 (34 occurrences) - Fixed output path from `path 'results'` to `path '.'` to avoid nested directories - Corrected default publish mode documentation (symlinks, not copies) ### Example Scripts (3 files) - Migrated hello-world.nf, hello-channels.nf, hello-workflow.nf - Added `publish:` sections to workflows - Added `output {}` blocks with correct path syntax - Removed all `publishDir` directives from processes ### Solution Files (14 workflow + 5 module files) - Updated all solution workflows to use new output syntax - Removed `publishDir` from all module files - Ensured all solutions follow modern Nextflow patterns ### Devcontainer Configs (3 files) - Updated NXF_VER from 25.04.3 to 25.10.0 in all devcontainer configs - Ensures consistent Nextflow version across development environments ## Testing - ✅ Tested hello-world.nf: outputs published correctly - ✅ Tested hello-channels.nf: single greeting published - ✅ Tested hello-workflow.nf: multiple outputs published correctly - All tests passed with Nextflow 25.10.0 ## Benefits - Students learn modern Nextflow syntax first - Clear migration path shown from legacy to new syntax - Centralized output management at workflow level - Aligned with Nextflow's future direction - All examples tested and verified working 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../codespaces-dev/devcontainer.json | 2 +- .devcontainer/devcontainer.json | 2 +- .devcontainer/local-dev/devcontainer.json | 2 +- docs/hello_nextflow/01_hello_world.md | 129 ++++++++++++++---- docs/hello_nextflow/02_hello_channels.md | 30 ++-- docs/hello_nextflow/03_hello_workflow.md | 75 ++++++++-- docs/hello_nextflow/04_hello_modules.md | 16 +-- docs/hello_nextflow/05_hello_containers.md | 15 +- docs/hello_nextflow/06_hello_config.md | 16 +-- hello-nextflow/hello-channels.nf | 13 +- hello-nextflow/hello-workflow.nf | 13 +- hello-nextflow/hello-world.nf | 11 +- .../solutions/1-hello-world/hello-world-3.nf | 13 +- .../solutions/1-hello-world/hello-world-4.nf | 13 +- .../2-hello-channels/hello-channels-1.nf | 13 +- .../2-hello-channels/hello-channels-2.nf | 13 +- .../2-hello-channels/hello-channels-3.nf | 13 +- .../2-hello-channels/hello-channels-4.nf | 13 +- .../3-hello-workflow/hello-workflow-1.nf | 21 ++- .../3-hello-workflow/hello-workflow-2.nf | 33 +++-- .../3-hello-workflow/hello-workflow-3.nf | 33 +++-- .../3-hello-workflow/hello-workflow-4.nf | 35 +++-- .../4-hello-modules/hello-modules-2.nf | 29 ++-- .../4-hello-modules/hello-modules-3.nf | 27 +++- .../modules/collectGreetings.nf | 2 - .../4-hello-modules/modules/convertToUpper.nf | 2 - .../4-hello-modules/modules/sayHello.nf | 2 - .../5-hello-containers/modules/cowpy.nf | 2 - .../solutions/6-hello-config/modules/cowpy.nf | 2 - 29 files changed, 408 insertions(+), 182 deletions(-) diff --git a/.devcontainer/codespaces-dev/devcontainer.json b/.devcontainer/codespaces-dev/devcontainer.json index d8a3a3bc9a..5474a4cc1b 100644 --- a/.devcontainer/codespaces-dev/devcontainer.json +++ b/.devcontainer/codespaces-dev/devcontainer.json @@ -32,7 +32,7 @@ // Nextflow installation version "NXF_HOME": "/workspaces/.nextflow", "NXF_EDGE": "0", - "NXF_VER": "25.04.3", + "NXF_VER": "25.10.0", // Other env vars "HOST_PROJECT_PATH": "/workspaces/training", "SHELL": "/bin/bash" // Ush bash diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json index be11210961..d291aa6184 100644 --- a/.devcontainer/devcontainer.json +++ b/.devcontainer/devcontainer.json @@ -12,7 +12,7 @@ // Nextflow installation version "NXF_HOME": "/workspaces/.nextflow", "NXF_EDGE": "0", - "NXF_VER": "25.04.3", + "NXF_VER": "25.10.0", // Other env vars "HOST_PROJECT_PATH": "/workspaces/training", "SHELL": "/bin/bash" // Ush bash diff --git a/.devcontainer/local-dev/devcontainer.json b/.devcontainer/local-dev/devcontainer.json index 750dbfb341..c967ee7179 100644 --- a/.devcontainer/local-dev/devcontainer.json +++ b/.devcontainer/local-dev/devcontainer.json @@ -33,7 +33,7 @@ // Nextflow installation version "NXF_HOME": "/workspaces/training/.nextflow", "NXF_EDGE": "0", - "NXF_VER": "25.04.3", + "NXF_VER": "25.10.0", // Other env vars "HOST_PROJECT_PATH": "/workspaces/training", "SHELL": "/bin/bash" // Ush bash diff --git a/docs/hello_nextflow/01_hello_world.md b/docs/hello_nextflow/01_hello_world.md index a9403ceea0..74ecf6112e 100644 --- a/docs/hello_nextflow/01_hello_world.md +++ b/docs/hello_nextflow/01_hello_world.md @@ -212,7 +212,7 @@ nextflow run hello-world.nf You console output should look something like this: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-world.nf` [goofy_torvalds] DSL2 - revision: c33d41f479 @@ -305,7 +305,7 @@ Learn how to manage your workflow executions conveniently. Knowing how to launch workflows and retrieve outputs is great, but you'll quickly find there are a few other aspects of workflow management that will make your life easier, especially if you're developing your own workflows. -Here we show you how to use the `publishDir` directive to store in an output folder all the main results from your pipeline run, the `resume` feature for when you need to re-launch the same workflow, and how to delete older work directories with `nextflow clean`. +Here we show you how to use workflow outputs to store in an output folder all the main results from your pipeline run, the `resume` feature for when you need to re-launch the same workflow, and how to delete older work directories with `nextflow clean`. ### 3.1. Publish outputs @@ -314,34 +314,74 @@ This is done on purpose; Nextflow is in control of this directory and we are not However, that makes it inconvenient to retrieve outputs that we care about. -Fortunately, Nextflow provides a way to manage this more conveniently, called the `publishDir` directive, which acts at the process level. -This directive tells Nextflow to publish the output(s) of the process to a designated output directory. By default, the outputs are published as symbolic links from the `work` directory. -It allows us to retrieve the desired output file without having to dig down into the work directory. +Fortunately, Nextflow provides a way to manage this more conveniently using **workflow outputs**. +Workflow outputs allow you to declare which outputs from your workflow should be published to a designated output directory. +This approach centralizes output publishing at the workflow level rather than at individual processes. -#### 3.1.1. Add a `publishDir` directive to the `sayHello` process +#### 3.1.1. Add workflow outputs to publish results -In the workflow script file `hello-world.nf`, make the following code modification: +In the workflow script file `hello-world.nf`, make the following code modification to add a `publish:` section to the workflow and an `output` block: === "After" - ```groovy title="hello-world.nf" linenums="6" hl_lines="3" - process sayHello { + ```groovy title="hello-world.nf" linenums="1" hl_lines="4 5 9 10 11 12" + #!/usr/bin/env nextflow - publishDir 'results', mode: 'copy' + process sayHello { output: path 'output.txt' + + script: + """ + echo 'Hello World!' > output.txt + """ + } + + workflow { + + // emit a greeting + ch_output = sayHello() + + publish: + greetings = ch_output + } + + output { + greetings { + path '.' + } + } ``` === "Before" - ```groovy title="hello-world.nf" linenums="6" + ```groovy title="hello-world.nf" linenums="1" + #!/usr/bin/env nextflow + process sayHello { output: path 'output.txt' + + script: + """ + echo 'Hello World!' > output.txt + """ + } + + workflow { + + // emit a greeting + sayHello() + } ``` +Let's break down what we added: + +1. **`publish:` section in the workflow**: This assigns the output channel from `sayHello()` to a named output called `greetings` +2. **`output` block**: This declares how the `greetings` output should be published. The `path '.'` means files will be published to the root of the output directory (which defaults to `results`) + #### 3.1.2. Run the workflow again Now run the modified workflow script: @@ -353,7 +393,7 @@ nextflow run hello-world.nf The log output should look very familiar: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-world.nf` [jovial_mayer] DSL2 - revision: 35bd3425e5 @@ -366,14 +406,46 @@ Our `output.txt` file is in this directory. If you check the contents it should match the output in the work subdirectory. This is how we publish results files outside of the working directories conveniently. -When you're dealing with very large files that you don't need to retain for long, you may prefer to set the `publishDir` directive to make a symbolic link to the file instead of copying it. -However, if you delete the work directory as part of a cleanup operation, you will lose access to the file, so always make sure you have actual copies of everything you care about before deleting anything. +By default, Nextflow creates symbolic links from the output directory to files in the work directory. +This is efficient because it doesn't duplicate files, but it means if you delete the work directory, you'll lose access to the outputs. +You can change this behavior to copy files instead using the `mode` directive in the output block or by setting `workflow.output.mode = 'copy'` in your configuration file. -!!! note +!!! tip + + You can change the output directory name by using the `-output-dir` command-line option: + ```bash + nextflow run hello-world.nf -output-dir my-results + ``` + +#### 3.1.3. Understanding workflow outputs vs. publishDir - A newer syntax option documented [here](https://www.nextflow.io/docs/latest/workflow.html#publishing-outputs) has been proposed to make it possible to declare and publish workflow-level outputs. - This will eventually make using `publishDir` at the process level redundant for completed pipelines. - However, we expect that `publishDir` will still remain very useful during pipeline development. +Workflow outputs are the modern approach to publishing results in Nextflow. +They provide several advantages: + +- **Centralized**: All publishing logic is in one place rather than scattered across process definitions +- **Flexible**: You can easily control what gets published from the workflow level +- **Cleaner modules**: Processes don't need to know where their outputs should be published + +However, you will still encounter the older `publishDir` directive when reading existing pipelines and working with nf-core modules. +The `publishDir` directive is applied at the process level and looks like this: + +```groovy +process sayHello { + + publishDir 'results', mode: 'copy' + + output: + path 'output.txt' + + script: + """ + echo 'Hello World!' > output.txt + """ +} +``` + +The `publishDir` directive is still supported but is being phased out in favor of workflow outputs. +Understanding both approaches will help you work with both modern and legacy Nextflow code. ### 3.2. Re-launch a workflow with `-resume` @@ -397,7 +469,7 @@ nextflow run hello-world.nf -resume The console output should look similar. ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-world.nf` [golden_cantor] DSL2 - revision: 35bd3425e5 @@ -411,7 +483,7 @@ Nextflow is literally pointing you to the previous execution and saying "I alrea !!! note - When your re-run a pipeline with `resume`, Nextflow does not overwrite any files written to a `publishDir` directory by any process call that was previously run successfully. + When you re-run a pipeline with `resume`, Nextflow does not overwrite any files written to the output directory by any process call that was previously run successfully. ### 3.3. Delete older work directories @@ -454,7 +526,8 @@ Removed /workspaces/training/hello-nextflow/work/a3/7be2fad5e71e5f49998f795677fd Deleting work subdirectories from past runs removes them from Nextflow's cache and deletes any outputs that were stored in those directories. That means it breaks Nextflow's ability to resume execution without re-running the corresponding processes. - You are responsible for saving any outputs that you care about or plan to rely on! If you're using the `publishDir` directive for that purpose, make sure to use the `copy` mode, not the `symlink` mode. + You are responsible for saving any outputs that you care about or plan to rely on by publishing them to the output directory! + Make sure your workflow outputs use the `copy` mode (the default), not the `symlink` mode. ### Takeaway @@ -489,11 +562,9 @@ In the process block, make the following code change: === "After" - ```groovy title="hello-world.nf" linenums="6" hl_lines="5 6" + ```groovy title="hello-world.nf" linenums="3" hl_lines="3 4" process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -503,11 +574,9 @@ In the process block, make the following code change: === "Before" - ```groovy title="hello-world.nf" linenums="6" + ```groovy title="hello-world.nf" linenums="3" process sayHello { - publishDir 'results', mode: 'copy' - output: path 'output.txt' ``` @@ -586,7 +655,7 @@ nextflow run hello-world.nf --greeting 'Bonjour le monde!' If you made all three edits correctly, you should get another successful execution: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-world.nf` [elated_lavoisier] DSL2 - revision: 7c031b42ea @@ -639,7 +708,7 @@ nextflow run hello-world.nf The console output should look the same. ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-world.nf` [determined_edison] DSL2 - revision: 3539118582 @@ -668,7 +737,7 @@ nextflow run hello-world.nf --greeting 'Konnichiwa!' The console output should look the same. ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-world.nf` [elegant_faraday] DSL2 - revision: 3539118582 diff --git a/docs/hello_nextflow/02_hello_channels.md b/docs/hello_nextflow/02_hello_channels.md index c8c5e179bf..c33535d251 100644 --- a/docs/hello_nextflow/02_hello_channels.md +++ b/docs/hello_nextflow/02_hello_channels.md @@ -37,7 +37,7 @@ nextflow run hello-channels.nf --greeting 'Hello Channels!' ``` ```console title="Output" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-channels.nf` [insane_lichterman] DSL2 - revision: c33d41f479 @@ -45,7 +45,7 @@ executor > local (1) [86/9efa08] sayHello | 1 of 1 ✔ ``` -As previously, you will find the output file named `output.txt` in the `results` directory (specified by the `publishDir` directive). +As previously, you will find the output file named `output.txt` in the `results` directory (specified by the workflow outputs). ```console title="results/output.txt" linenums="1" Hello Channels! @@ -151,7 +151,7 @@ nextflow run hello-channels.nf If you made both edits correctly, you should get another successful execution: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-channels.nf` [nice_heisenberg] DSL2 - revision: 41b4aeb7e9 @@ -221,7 +221,7 @@ nextflow run hello-channels.nf It certainly seems to run just fine: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-channels.nf` [suspicious_lamport] DSL2 - revision: 778deadaea @@ -246,7 +246,7 @@ nextflow run hello-channels.nf -ansi-log false This time we see all three process runs and their associated work subdirectories listed in the output: ```console title="Output" linenums="1" -N E X T F L O W ~ version 25.04.3 +N E X T F L O W ~ version 25.10.0 Launching `hello-channels.nf` [pensive_poitras] DSL2 - revision: 778deadaea [76/f61695] Submitted process > sayHello (1) [6e/d12e35] Submitted process > sayHello (3) @@ -282,7 +282,7 @@ You may recall that we hardcoded the output file name for the `sayHello` process You can check the work subdirectories for each of the three processes; each of them contains a file called `output.txt` as expected. As long as the output files stay there, isolated from the other processes, that is okay. -But when the `publishDir` directive copies each of them to the same `results` directory, whichever got copied there first gets overwritten by the next one, and so on. +But when the workflow outputs publish each of them to the same `results` directory, whichever got published there first gets overwritten by the next one, and so on. ### 2.2. Ensure the output file names will be unique @@ -299,11 +299,9 @@ In the process block, make the following code changes: === "After" - ```groovy title="hello-channels.nf" linenums="6" hl_lines="9 13" + ```groovy title="hello-channels.nf" linenums="6" hl_lines="7 11" process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -322,8 +320,6 @@ In the process block, make the following code changes: ```groovy title="hello-channels.nf" linenums="6" process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -356,7 +352,7 @@ nextflow run hello-channels.nf Reverting back to the summary view, the output looks like this again: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-channels.nf` [astonishing_bell] DSL2 - revision: f57ff44a69 @@ -480,7 +476,7 @@ nextflow run hello-channels.nf Oh no! Nextflow throws an error that starts like this: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-channels.nf` [friendly_koch] DSL2 - revision: 97256837a7 @@ -582,7 +578,7 @@ nextflow run hello-channels.nf This time it works AND gives us the additional insight into what the contents of the channel look like before and after we run the `flatten()` operator: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-channels.nf` [tiny_elion] DSL2 - revision: 1d834f23d2 @@ -699,7 +695,7 @@ nextflow run hello-channels.nf Oh no, it doesn't work. Here's the start of the console output and error message: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-channels.nf` [adoring_bhabha] DSL2 - revision: 8ce25edc39 @@ -766,7 +762,7 @@ nextflow run hello-channels.nf Interestingly, this fails too, but with a different error. The console output and error starts like this: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-channels.nf` [stoic_ride] DSL2 - revision: a0e5de507e @@ -854,7 +850,7 @@ nextflow run hello-channels.nf This time it should run without error. ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-channels.nf` [tiny_heisenberg] DSL2 - revision: 845b471427 diff --git a/docs/hello_nextflow/03_hello_workflow.md b/docs/hello_nextflow/03_hello_workflow.md index d5bcd340cd..868f5033d9 100644 --- a/docs/hello_nextflow/03_hello_workflow.md +++ b/docs/hello_nextflow/03_hello_workflow.md @@ -42,7 +42,7 @@ nextflow run hello-workflow.nf ``` ```console title="Output" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-workflow.nf` [stupefied_sammet] DSL2 - revision: b9e466930b @@ -50,7 +50,7 @@ executor > local (3) [2a/324ce6] sayHello (3) | 3 of 3 ✔ ``` -As previously, you will find the output files in the `results` directory (specified by the `publishDir` directive). +As previously, you will find the output files in the `results` directory (specified by the workflow outputs). ```console title="Directory contents" results @@ -112,8 +112,6 @@ Add the following process definition to the workflow script: */ process convertToUpper { - publishDir 'results', mode: 'copy' - input: path input_file @@ -199,7 +197,7 @@ nextflow run hello-workflow.nf -resume You should see the following output: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-workflow.nf` [disturbed_darwin] DSL2 - revision: 4e252c048f @@ -224,7 +222,7 @@ The output of the first process is in there because Nextflow staged it there in However, it is actually a symbolic link pointing to the the original file in the subdirectory of the first process call. By default, when running on a single machine as we're doing here, Nextflow uses symbolic links rather than copies to stage input and intermediate files. -You'll also find the final outputs in the `results` directory since we used the `publishDir` directive in the second process too. +You'll also find the final outputs in the `results` directory, as we'll configure next. ```console title="Directory contents" results @@ -241,6 +239,61 @@ Nextflow did the hard work of handling individual input and output files and pas This is one of the reasons Nextflow channels are so powerful: they take care of the busywork involved in connecting workflow steps together. +### 1.6. Update workflow outputs to publish both processes + +Now let's update our workflow outputs to publish the results from both processes. + +In the workflow and output blocks, make the following code change: + +=== "After" + + ```groovy title="hello-workflow.nf" linenums="53" hl_lines="4 7 8 9 10 11 12 13 14 15 16 17" + // emit a greeting + ch_hello = sayHello(greeting_ch) + + // convert the greeting to uppercase + ch_upper = convertToUpper(ch_hello) + + publish: + greetings = ch_hello + uppercase = ch_upper + } + + output { + greetings { + path '.' + } + uppercase { + path '.' + } + } + ``` + +=== "Before" + + ```groovy title="hello-workflow.nf" linenums="53" + // emit a greeting + ch_output = sayHello(greeting_ch) + + publish: + greetings = ch_output + } + + output { + greetings { + path '.' + } + } + ``` + +Here we: + +1. Assigned the output of `sayHello()` to `ch_hello` and `convertToUpper()` to `ch_upper` +2. Added both channels to the `publish:` section +3. Declared both outputs in the `output` block, with both publishing to the `results` directory + +Now when you run the workflow, both the original greetings and the uppercase versions will be published to the `results` directory. + ### Takeaway You know how to add a second step that takes the output of the first step as input. @@ -297,8 +350,6 @@ Add the following process definition to the workflow script: */ process collectGreetings { - publishDir 'results', mode: 'copy' - input: ??? @@ -422,7 +473,7 @@ nextflow run hello-workflow.nf -resume It runs successfully, including the third step: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-workflow.nf` [mad_gilbert] DSL2 - revision: 6acfd5e28d @@ -513,7 +564,7 @@ nextflow run hello-workflow.nf -resume It runs successfully, although the log output may look a little messier than this (we cleaned it up for readability). ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-workflow.nf` [soggy_franklin] DSL2 - revision: bc8e1b2726 @@ -693,7 +744,7 @@ nextflow run hello-workflow.nf -resume --batch trio It runs successfully: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-workflow.nf` [confident_rutherford] DSL2 - revision: bc58af409c @@ -844,7 +895,7 @@ nextflow run hello-workflow.nf -resume --batch trio This runs successfully: ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-workflow.nf` [evil_sinoussi] DSL2 - revision: eeca64cdb1 diff --git a/docs/hello_nextflow/04_hello_modules.md b/docs/hello_nextflow/04_hello_modules.md index cc2476a6b4..d6af375fc4 100644 --- a/docs/hello_nextflow/04_hello_modules.md +++ b/docs/hello_nextflow/04_hello_modules.md @@ -39,7 +39,7 @@ nextflow run hello-modules.nf ``` ```console title="Output" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-modules.nf` [festering_nobel] DSL2 - revision: eeca64cdb1 @@ -50,7 +50,7 @@ executor > local (7) There were 3 greetings in this batch ``` -As previously, you will find the output files in the `results` directory (specified by the `publishDir` directive). +As previously, you will find the output files in the `results` directory (specified by the workflow outputs). ```console title="Directory contents" results @@ -117,8 +117,6 @@ Copy the whole process definition over from the workflow file to the module file */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -170,7 +168,7 @@ nextflow run hello-modules.nf -resume This runs quickly very quickly because everything is cached. ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-modules.nf` [romantic_poisson] DSL2 - revision: 96edfa9ad3 @@ -216,8 +214,6 @@ Copy the whole process definition over from the workflow file to the module file */ process convertToUpper { - publishDir 'results', mode: 'copy' - input: path input_file @@ -267,7 +263,7 @@ nextflow run hello-modules.nf -resume This should still produce the same output as previously. ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-modules.nf` [nauseous_heisenberg] DSL2 - revision: a04a9f2da0 @@ -303,8 +299,6 @@ Copy the whole process definition over from the workflow file to the module file */ process collectGreetings { - publishDir 'results', mode: 'copy' - input: path input_files val batch_name @@ -359,7 +353,7 @@ nextflow run hello-modules.nf -resume This should still produce the same output as previously. ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-modules.nf` [friendly_coulomb] DSL2 - revision: 7aa2b9bc0f diff --git a/docs/hello_nextflow/05_hello_containers.md b/docs/hello_nextflow/05_hello_containers.md index eaaade723a..cd4e1f5227 100644 --- a/docs/hello_nextflow/05_hello_containers.md +++ b/docs/hello_nextflow/05_hello_containers.md @@ -40,7 +40,7 @@ nextflow run hello-containers.nf This should produce the following output: ```console title="Output" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-containers.nf` [tender_becquerel] DSL2 - revision: f7cat8e223 @@ -50,7 +50,7 @@ executor > local (7) [7d/f7961c] collectGreetings [100%] 1 of 1 ✔ ``` -As previously, you will find the output files in the `results` directory (specified by the `publishDir` directive). +As previously, you will find the output files in the `results` directory (specified by the workflow outputs). ```console title="Directory contents" results @@ -378,8 +378,6 @@ We can model our `cowpy` process on the other processes we've written previously // Generate ASCII art with cowpy process cowpy { - publishDir 'results', mode: 'copy' - input: path input_file val character @@ -500,7 +498,7 @@ nextflow run hello-containers.nf -resume Oh no, there's an error! ```console title="Output" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-containers.nf` [special_lovelace] DSL2 - revision: 028a841db1 @@ -545,10 +543,9 @@ Edit the `cowpy.nf` module to add the `container` directive to the process defin === "After" - ```groovy title="modules/cowpy.nf" linenums="4" hl_lines="4" + ```groovy title="modules/cowpy.nf" linenums="4" hl_lines="3" process cowpy { - publishDir 'containers/results', mode: 'copy' container 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273' ``` @@ -556,8 +553,6 @@ Edit the `cowpy.nf` module to add the `container` directive to the process defin ```groovy title="modules/cowpy.nf" linenums="4" process cowpy { - - publishDir 'containers/results', mode: 'copy' ``` This tells Nextflow that if the use of Docker is enabled, it should use the container image specified here to execute the process. @@ -601,7 +596,7 @@ nextflow run hello-containers.nf -resume This time it does indeed work. ```console title="Output" linenums="1" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-containers.nf` [elegant_brattain] DSL2 - revision: 028a841db1 diff --git a/docs/hello_nextflow/06_hello_config.md b/docs/hello_nextflow/06_hello_config.md index a8c7225811..a7ff60e30b 100644 --- a/docs/hello_nextflow/06_hello_config.md +++ b/docs/hello_nextflow/06_hello_config.md @@ -44,7 +44,7 @@ nextflow run hello-config.nf ``` ```console title="Output" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-config.nf` [reverent_heisenberg] DSL2 - revision: 028a841db1 @@ -115,13 +115,11 @@ Now we add the URI to the `cowpy` process definition using the `conda` directive === "After" - ```console title="modules/cowpy.nf" linenums="4" hl_lines="4" + ```console title="modules/cowpy.nf" linenums="4" hl_lines="3" process cowpy { container 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273' conda 'conda-forge::cowpy==1.1.5' - - publishDir 'results', mode: 'copy' ``` === "Before" @@ -130,8 +128,6 @@ Now we add the URI to the `cowpy` process definition using the `conda` directive process cowpy { container 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273' - - publishDir 'results', mode: 'copy' ``` To be clear, we're not _replacing_ the `docker` directive, we're _adding_ an alternative option. @@ -147,7 +143,7 @@ nextflow run hello-config.nf This should work without issue. ```console title="Output" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-config.nf` [trusting_lovelace] DSL2 - revision: 028a841db1 @@ -362,7 +358,7 @@ nextflow run hello-config.nf -params-file test-params.json It works! And as expected, this produces the same outputs as previously. ```console title="Output" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-config.nf` [disturbed_sammet] DSL2 - revision: ede9037d02 @@ -511,7 +507,7 @@ nextflow run hello-config.nf -profile my_laptop This still produces the following output: ``` - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-config.nf` [gigantic_brazil] DSL2 - revision: ede9037d02 @@ -593,7 +589,7 @@ nextflow run hello-config.nf -profile my_laptop,test This should produce the following: ```console title="Output" - N E X T F L O W ~ version 25.04.3 + N E X T F L O W ~ version 25.10.0 Launching `hello-config.nf` [gigantic_brazil] DSL2 - revision: ede9037d02 diff --git a/hello-nextflow/hello-channels.nf b/hello-nextflow/hello-channels.nf index 6236eea21e..9610a01a71 100644 --- a/hello-nextflow/hello-channels.nf +++ b/hello-nextflow/hello-channels.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -27,5 +25,14 @@ params.greeting = 'Holà mundo!' workflow { // emit a greeting - sayHello(params.greeting) + ch_output = sayHello(params.greeting) + + publish: + greetings = ch_output +} + +output { + greetings { + path '.' + } } diff --git a/hello-nextflow/hello-workflow.nf b/hello-nextflow/hello-workflow.nf index 0f54be80ce..0660da8311 100644 --- a/hello-nextflow/hello-workflow.nf +++ b/hello-nextflow/hello-workflow.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -32,5 +30,14 @@ workflow { .map { line -> line[0] } // emit a greeting - sayHello(greeting_ch) + ch_output = sayHello(greeting_ch) + + publish: + greetings = ch_output +} + +output { + greetings { + path '.' + } } diff --git a/hello-nextflow/hello-world.nf b/hello-nextflow/hello-world.nf index 4f672da2b5..c70aeae5b8 100644 --- a/hello-nextflow/hello-world.nf +++ b/hello-nextflow/hello-world.nf @@ -17,5 +17,14 @@ process sayHello { workflow { // emit a greeting - sayHello() + ch_output = sayHello() + + publish: + greetings = ch_output +} + +output { + greetings { + path '.' + } } diff --git a/hello-nextflow/solutions/1-hello-world/hello-world-3.nf b/hello-nextflow/solutions/1-hello-world/hello-world-3.nf index f2e4b33448..c70aeae5b8 100644 --- a/hello-nextflow/solutions/1-hello-world/hello-world-3.nf +++ b/hello-nextflow/solutions/1-hello-world/hello-world-3.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - output: path 'output.txt' @@ -19,5 +17,14 @@ process sayHello { workflow { // emit a greeting - sayHello() + ch_output = sayHello() + + publish: + greetings = ch_output +} + +output { + greetings { + path '.' + } } diff --git a/hello-nextflow/solutions/1-hello-world/hello-world-4.nf b/hello-nextflow/solutions/1-hello-world/hello-world-4.nf index 6236eea21e..9610a01a71 100644 --- a/hello-nextflow/solutions/1-hello-world/hello-world-4.nf +++ b/hello-nextflow/solutions/1-hello-world/hello-world-4.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -27,5 +25,14 @@ params.greeting = 'Holà mundo!' workflow { // emit a greeting - sayHello(params.greeting) + ch_output = sayHello(params.greeting) + + publish: + greetings = ch_output +} + +output { + greetings { + path '.' + } } diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf index 6190b616ae..15a892e28f 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -30,5 +28,14 @@ workflow { greeting_ch = Channel.of('Hello Channels!') // emit a greeting - sayHello(greeting_ch) + ch_output = sayHello(greeting_ch) + + publish: + greetings = ch_output +} + +output { + greetings { + path '.' + } } diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf index 5e7a2d50fc..240544315a 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -30,5 +28,14 @@ workflow { greeting_ch = Channel.of('Hello','Bonjour','Holà') // emit a greeting - sayHello(greeting_ch) + ch_output = sayHello(greeting_ch) + + publish: + greetings = ch_output +} + +output { + greetings { + path '.' + } } diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf index 49de2cb657..b5cdfb5eba 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -35,5 +33,14 @@ workflow { .view { "After flatten: $it" } // emit a greeting - sayHello(greeting_ch) + ch_output = sayHello(greeting_ch) + + publish: + greetings = ch_output +} + +output { + greetings { + path '.' + } } diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf index 449366b810..c2569add8a 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -37,5 +35,14 @@ workflow { .view { "After map: $it" } // emit a greeting - sayHello(greeting_ch) + ch_output = sayHello(greeting_ch) + + publish: + greetings = ch_output +} + +output { + greetings { + path '.' + } } diff --git a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf index 9516d600f7..82104db3d1 100644 --- a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf +++ b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -24,8 +22,6 @@ process sayHello { */ process convertToUpper { - publishDir 'results', mode: 'copy' - input: path input_file @@ -51,8 +47,21 @@ workflow { .map { line -> line[0] } // emit a greeting - sayHello(greeting_ch) + ch_hello = sayHello(greeting_ch) // convert the greeting to uppercase - convertToUpper(sayHello.out) + ch_upper = convertToUpper(ch_hello) + + publish: + greetings = ch_hello + uppercase = ch_upper +} + +output { + greetings { + path '.' + } + uppercase { + path '.' + } } diff --git a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf index f517924175..d28d36bb7d 100644 --- a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf +++ b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -24,8 +22,6 @@ process sayHello { */ process convertToUpper { - publishDir 'results', mode: 'copy' - input: path input_file @@ -43,8 +39,6 @@ process convertToUpper { */ process collectGreetings { - publishDir 'results', mode: 'copy' - input: path input_files @@ -70,15 +64,32 @@ workflow { .map { line -> line[0] } // emit a greeting - sayHello(greeting_ch) + ch_hello = sayHello(greeting_ch) // convert the greeting to uppercase - convertToUpper(sayHello.out) + ch_upper = convertToUpper(ch_hello) // collect all the greetings into one file - collectGreetings(convertToUpper.out.collect()) + ch_collected = collectGreetings(ch_upper.collect()) // optional view statements - convertToUpper.out.view { "Before collect: $it" } - convertToUpper.out.collect().view { "After collect: $it" } + ch_upper.view { "Before collect: $it" } + ch_upper.collect().view { "After collect: $it" } + + publish: + greetings = ch_hello + uppercase = ch_upper + collected = ch_collected +} + +output { + greetings { + path '.' + } + uppercase { + path '.' + } + collected { + path '.' + } } diff --git a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf index 65ac85ce0d..3b3b9f6c1f 100644 --- a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf +++ b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -24,8 +22,6 @@ process sayHello { */ process convertToUpper { - publishDir 'results', mode: 'copy' - input: path input_file @@ -43,8 +39,6 @@ process convertToUpper { */ process collectGreetings { - publishDir 'results', mode: 'copy' - input: path input_files val batch_name @@ -72,15 +66,32 @@ workflow { .map { line -> line[0] } // emit a greeting - sayHello(greeting_ch) + ch_hello = sayHello(greeting_ch) // convert the greeting to uppercase - convertToUpper(sayHello.out) + ch_upper = convertToUpper(ch_hello) // collect all the greetings into one file - collectGreetings(convertToUpper.out.collect(), params.batch) + ch_collected = collectGreetings(ch_upper.collect(), params.batch) // optional view statements - convertToUpper.out.view { "Before collect: $it" } - convertToUpper.out.collect().view { "After collect: $it" } + ch_upper.view { "Before collect: $it" } + ch_upper.collect().view { "After collect: $it" } + + publish: + greetings = ch_hello + uppercase = ch_upper + collected = ch_collected +} + +output { + greetings { + path '.' + } + uppercase { + path '.' + } + collected { + path '.' + } } diff --git a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-4.nf b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-4.nf index 92af832e72..715eab412e 100644 --- a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-4.nf +++ b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-4.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -24,8 +22,6 @@ process sayHello { */ process convertToUpper { - publishDir 'results', mode: 'copy' - input: path input_file @@ -43,8 +39,6 @@ process convertToUpper { */ process collectGreetings { - publishDir 'results', mode: 'copy' - input: path input_files val batch_name @@ -74,18 +68,35 @@ workflow { .map { line -> line[0] } // emit a greeting - sayHello(greeting_ch) + ch_hello = sayHello(greeting_ch) // convert the greeting to uppercase - convertToUpper(sayHello.out) + ch_upper = convertToUpper(ch_hello) // collect all the greetings into one file - collectGreetings(convertToUpper.out.collect(), params.batch) + ch_collected = collectGreetings(ch_upper.collect(), params.batch) // emit a message about the size of the batch - collectGreetings.out.count.view { "There were $it greetings in this batch" } + ch_collected.count.view { "There were $it greetings in this batch" } // optional view statements - //convertToUpper.out.view { "Before collect: $it" } - //convertToUpper.out.collect().view { "After collect: $it" } + //ch_upper.view { "Before collect: $it" } + //ch_upper.collect().view { "After collect: $it" } + + publish: + greetings = ch_hello + uppercase = ch_upper + collected = ch_collected.outfile +} + +output { + greetings { + path '.' + } + uppercase { + path '.' + } + collected { + path '.' + } } diff --git a/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf b/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf index 0d384c4796..ac9eaf5e40 100644 --- a/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf +++ b/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf @@ -5,8 +5,6 @@ */ process convertToUpper { - publishDir 'results', mode: 'copy' - input: path input_file @@ -24,8 +22,6 @@ process convertToUpper { */ process collectGreetings { - publishDir 'results', mode: 'copy' - input: path input_files val batch_name @@ -58,14 +54,31 @@ workflow { .map { line -> line[0] } // emit a greeting - sayHello(greeting_ch) + ch_hello = sayHello(greeting_ch) // convert the greeting to uppercase - convertToUpper(sayHello.out) + ch_upper = convertToUpper(ch_hello) // collect all the greetings into one file - collectGreetings(convertToUpper.out.collect(), params.batch) + ch_collected = collectGreetings(ch_upper.collect(), params.batch) // emit a message about the size of the batch - collectGreetings.out.count.view { "There were $it greetings in this batch" } + ch_collected.count.view { "There were $it greetings in this batch" } + + publish: + greetings = ch_hello + uppercase = ch_upper + collected = ch_collected.outfile +} + +output { + greetings { + path '.' + } + uppercase { + path '.' + } + collected { + path '.' + } } diff --git a/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf b/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf index e34edebed0..c0e83e7b29 100644 --- a/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf +++ b/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf @@ -5,8 +5,6 @@ */ process collectGreetings { - publishDir 'results', mode: 'copy' - input: path input_files val batch_name @@ -40,14 +38,31 @@ workflow { .map { line -> line[0] } // emit a greeting - sayHello(greeting_ch) + ch_hello = sayHello(greeting_ch) // convert the greeting to uppercase - convertToUpper(sayHello.out) + ch_upper = convertToUpper(ch_hello) // collect all the greetings into one file - collectGreetings(convertToUpper.out.collect(), params.batch) + ch_collected = collectGreetings(ch_upper.collect(), params.batch) // emit a message about the size of the batch - collectGreetings.out.count.view { "There were $it greetings in this batch" } + ch_collected.count.view { "There were $it greetings in this batch" } + + publish: + greetings = ch_hello + uppercase = ch_upper + collected = ch_collected.outfile +} + +output { + greetings { + path '.' + } + uppercase { + path '.' + } + collected { + path '.' + } } diff --git a/hello-nextflow/solutions/4-hello-modules/modules/collectGreetings.nf b/hello-nextflow/solutions/4-hello-modules/modules/collectGreetings.nf index 849bba4b6e..09b8fa1fce 100644 --- a/hello-nextflow/solutions/4-hello-modules/modules/collectGreetings.nf +++ b/hello-nextflow/solutions/4-hello-modules/modules/collectGreetings.nf @@ -3,8 +3,6 @@ */ process collectGreetings { - publishDir 'results', mode: 'copy' - input: path input_files val batch_name diff --git a/hello-nextflow/solutions/4-hello-modules/modules/convertToUpper.nf b/hello-nextflow/solutions/4-hello-modules/modules/convertToUpper.nf index b2689e8e9c..20d0b0885a 100644 --- a/hello-nextflow/solutions/4-hello-modules/modules/convertToUpper.nf +++ b/hello-nextflow/solutions/4-hello-modules/modules/convertToUpper.nf @@ -5,8 +5,6 @@ */ process convertToUpper { - publishDir 'results', mode: 'copy' - input: path input_file diff --git a/hello-nextflow/solutions/4-hello-modules/modules/sayHello.nf b/hello-nextflow/solutions/4-hello-modules/modules/sayHello.nf index 6005ad54c9..e2cd73ae85 100644 --- a/hello-nextflow/solutions/4-hello-modules/modules/sayHello.nf +++ b/hello-nextflow/solutions/4-hello-modules/modules/sayHello.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting diff --git a/hello-nextflow/solutions/5-hello-containers/modules/cowpy.nf b/hello-nextflow/solutions/5-hello-containers/modules/cowpy.nf index 60b739b625..07a06c70f8 100644 --- a/hello-nextflow/solutions/5-hello-containers/modules/cowpy.nf +++ b/hello-nextflow/solutions/5-hello-containers/modules/cowpy.nf @@ -3,8 +3,6 @@ // Generate ASCII art with cowpy (https://github.com/jeffbuttars/cowpy) process cowpy { - publishDir 'results', mode: 'copy' - container 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273' input: diff --git a/hello-nextflow/solutions/6-hello-config/modules/cowpy.nf b/hello-nextflow/solutions/6-hello-config/modules/cowpy.nf index 2bc7ed612b..7c850fbd4c 100644 --- a/hello-nextflow/solutions/6-hello-config/modules/cowpy.nf +++ b/hello-nextflow/solutions/6-hello-config/modules/cowpy.nf @@ -3,8 +3,6 @@ // Generate ASCII art with cowpy (https://github.com/jeffbuttars/cowpy) process cowpy { - publishDir 'results', mode: 'copy' - container 'community.wave.seqera.io/library/cowpy:1.1.5--3db457ae1977a273' conda 'conda-forge::cowpy==1.1.5' From 08bff170e95842444cb08799f0e95ea7d756a705 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Fri, 24 Oct 2025 16:45:56 +0200 Subject: [PATCH 02/17] Fix highlight --- docs/hello_nextflow/01_hello_world.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/hello_nextflow/01_hello_world.md b/docs/hello_nextflow/01_hello_world.md index 74ecf6112e..ed7369a022 100644 --- a/docs/hello_nextflow/01_hello_world.md +++ b/docs/hello_nextflow/01_hello_world.md @@ -324,7 +324,7 @@ In the workflow script file `hello-world.nf`, make the following code modificati === "After" - ```groovy title="hello-world.nf" linenums="1" hl_lines="4 5 9 10 11 12" + ```groovy title="hello-world.nf" linenums="1" hl_lines="17-27" #!/usr/bin/env nextflow process sayHello { @@ -356,7 +356,7 @@ In the workflow script file `hello-world.nf`, make the following code modificati === "Before" - ```groovy title="hello-world.nf" linenums="1" + ```groovy title="hello-world.nf" linenums="1" hl_lines="17" #!/usr/bin/env nextflow process sayHello { From 36b2bbed726cbf6e9fe73372453bd2ed030d0074 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Fri, 24 Oct 2025 16:48:43 +0200 Subject: [PATCH 03/17] Fix highlight --- docs/hello_nextflow/01_hello_world.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hello_nextflow/01_hello_world.md b/docs/hello_nextflow/01_hello_world.md index ed7369a022..322abbd37d 100644 --- a/docs/hello_nextflow/01_hello_world.md +++ b/docs/hello_nextflow/01_hello_world.md @@ -356,7 +356,7 @@ In the workflow script file `hello-world.nf`, make the following code modificati === "Before" - ```groovy title="hello-world.nf" linenums="1" hl_lines="17" + ```groovy title="hello-world.nf" linenums="1" hl_lines="17-18" #!/usr/bin/env nextflow process sayHello { From f01e98de7546b0fc6a96aede62df0b0449338489 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Fri, 24 Oct 2025 16:56:18 +0200 Subject: [PATCH 04/17] Fix workflow consistency in section 4.1.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Section 4.1.3 was showing sayHello() without assigning to ch_output, which was inconsistent with the workflow outputs pattern taught in section 3.1. Updated both 'After' and 'Before' code blocks to include ch_output assignment for consistency. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/hello_nextflow/01_hello_world.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/hello_nextflow/01_hello_world.md b/docs/hello_nextflow/01_hello_world.md index 322abbd37d..56160fd7f8 100644 --- a/docs/hello_nextflow/01_hello_world.md +++ b/docs/hello_nextflow/01_hello_world.md @@ -632,14 +632,14 @@ In the workflow block, make the following code change: ```groovy title="hello-world.nf" linenums="24" hl_lines="2" // emit a greeting - sayHello(params.greeting) + ch_output = sayHello(params.greeting) ``` === "Before" ```groovy title="hello-world.nf" linenums="24" // emit a greeting - sayHello() + ch_output = sayHello() ``` This tells Nextflow to run the `sayHello` process on the value provided through the `--greeting` parameter. From 173fb26deab4c215f7df183590704467520a9f61 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Fri, 24 Oct 2025 17:05:03 +0200 Subject: [PATCH 05/17] Fix workflow consistency in lesson 02 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updated code examples to consistently show ch_output assignment when calling sayHello(), maintaining consistency with the pattern taught in lesson 01 section 3.1. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/hello_nextflow/02_hello_channels.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/hello_nextflow/02_hello_channels.md b/docs/hello_nextflow/02_hello_channels.md index c33535d251..69289262e4 100644 --- a/docs/hello_nextflow/02_hello_channels.md +++ b/docs/hello_nextflow/02_hello_channels.md @@ -88,7 +88,7 @@ In the workflow block, add the channel factory code: greeting_ch = Channel.of('Hello Channels!') // emit a greeting - sayHello(params.greeting) + ch_output = sayHello(params.greeting) } ``` @@ -98,7 +98,7 @@ In the workflow block, add the channel factory code: workflow { // emit a greeting - sayHello(params.greeting) + ch_output = sayHello(params.greeting) } ``` @@ -119,7 +119,7 @@ In the workflow block, make the following code change: greeting_ch = Channel.of('Hello Channels!') // emit a greeting - sayHello(greeting_ch) + ch_output = sayHello(greeting_ch) } ``` @@ -132,7 +132,7 @@ In the workflow block, make the following code change: greeting_ch = Channel.of('Hello Channels!') // emit a greeting - sayHello(params.greeting) + ch_output = sayHello(params.greeting) } ``` From f8f50d3c9d80a9a3b9ff3f24d46003a71f00948c Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Fri, 24 Oct 2025 17:06:27 +0200 Subject: [PATCH 06/17] Clarify transition from .out to named channels in lesson 03 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Enhanced the explanation in section 1.6 to clearly explain WHY we switch from using sayHello.out notation to named channel assignments (ch_hello, ch_upper). This helps students understand that named channels are needed for the publish section and are generally cleaner when working with workflow outputs. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/hello_nextflow/03_hello_workflow.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/hello_nextflow/03_hello_workflow.md b/docs/hello_nextflow/03_hello_workflow.md index 868f5033d9..7a30f0c195 100644 --- a/docs/hello_nextflow/03_hello_workflow.md +++ b/docs/hello_nextflow/03_hello_workflow.md @@ -286,11 +286,13 @@ In the workflow and output blocks, make the following code change: } ``` -Here we: +Here we made several changes: -1. Assigned the output of `sayHello()` to `ch_hello` and `convertToUpper()` to `ch_upper` -2. Added both channels to the `publish:` section -3. Declared both outputs in the `output` block, with both publishing to the `results` directory +1. **Assigned outputs to named channels**: Changed from using `sayHello.out` to `ch_hello = sayHello()` and similarly for `convertToUpper()`. This gives us named channels that we can reference in the publish section. +2. **Added both channels to the `publish:` section**: This tells Nextflow which outputs we want to publish. +3. **Declared both outputs in the `output` block**: Specifies that both should be published to the root of the output directory. + +By assigning process outputs to named channels, we can easily reference them when declaring what to publish. This is cleaner than using the `.out` notation, especially when working with multiple outputs. Now when you run the workflow, both the original greetings and the uppercase versions will be published to the `results` directory. From cc6e8a933f1411f07426dff5d66c161c560931e1 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Fri, 24 Oct 2025 17:13:45 +0200 Subject: [PATCH 07/17] Fix workflow consistency in solution files MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updated hello-modules-4.nf and hello-containers-2.nf to use consistent workflow patterns: - Changed from .out notation to named channel assignments - Added publish: and output {} blocks - Ensures all solution files follow the same pattern as teaching materials Tested: hello-modules-4.nf runs successfully and publishes outputs correctly 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../4-hello-modules/hello-modules-4.nf | 25 ++++++++++++--- .../5-hello-containers/hello-containers-2.nf | 31 ++++++++++++++++--- 2 files changed, 47 insertions(+), 9 deletions(-) diff --git a/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf b/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf index 8764606bf7..cc533e4b29 100644 --- a/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf +++ b/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf @@ -19,14 +19,31 @@ workflow { .map { line -> line[0] } // emit a greeting - sayHello(greeting_ch) + ch_hello = sayHello(greeting_ch) // convert the greeting to uppercase - convertToUpper(sayHello.out) + ch_upper = convertToUpper(ch_hello) // collect all the greetings into one file - collectGreetings(convertToUpper.out.collect(), params.batch) + ch_collected = collectGreetings(ch_upper.collect(), params.batch) // emit a message about the size of the batch - collectGreetings.out.count.view { "There were $it greetings in this batch" } + ch_collected.count.view { "There were $it greetings in this batch" } + + publish: + greetings = ch_hello + uppercase = ch_upper + collected = ch_collected.outfile +} + +output { + greetings { + path '.' + } + uppercase { + path '.' + } + collected { + path '.' + } } diff --git a/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf b/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf index f187831d1c..1a0c6d365e 100644 --- a/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf +++ b/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf @@ -21,17 +21,38 @@ workflow { .map { line -> line[0] } // emit a greeting - sayHello(greeting_ch) + ch_hello = sayHello(greeting_ch) // convert the greeting to uppercase - convertToUpper(sayHello.out) + ch_upper = convertToUpper(ch_hello) // collect all the greetings into one file - collectGreetings(convertToUpper.out.collect(), params.batch) + ch_collected = collectGreetings(ch_upper.collect(), params.batch) // emit a message about the size of the batch - collectGreetings.out.count.view { "There were $it greetings in this batch" } + ch_collected.count.view { "There were $it greetings in this batch" } // generate ASCII art of the greetings with cowpy - cowpy(collectGreetings.out.outfile, params.character) + ch_cowpy = cowpy(ch_collected.outfile, params.character) + + publish: + greetings = ch_hello + uppercase = ch_upper + collected = ch_collected.outfile + ascii_art = ch_cowpy +} + +output { + greetings { + path '.' + } + uppercase { + path '.' + } + collected { + path '.' + } + ascii_art { + path '.' + } } From 2f03222ce8a506e629f1fa5dd43dec0e94b5b7c1 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Fri, 24 Oct 2025 17:30:21 +0200 Subject: [PATCH 08/17] fix some indents --- docs/hello_nextflow/02_hello_channels.md | 36 ++++++++++++------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/hello_nextflow/02_hello_channels.md b/docs/hello_nextflow/02_hello_channels.md index 69289262e4..f8c3f9ad21 100644 --- a/docs/hello_nextflow/02_hello_channels.md +++ b/docs/hello_nextflow/02_hello_channels.md @@ -514,7 +514,7 @@ In the workflow block, make the following code change: ```groovy title="hello-channels.nf" linenums="31" hl_lines="3" // create a channel for inputs greeting_ch = Channel.of(greetings_array) - .flatten() + .flatten() ``` === "Before" @@ -539,9 +539,9 @@ In the workflow block, make the following code change: ```groovy title="hello-channels.nf" linenums="31" hl_lines="3-5" // create a channel for inputs greeting_ch = Channel.of(greetings_array) - .view { greeting -> "Before flatten: $greeting" } - .flatten() - .view { greeting -> "After flatten: $greeting" } + .view { greeting -> "Before flatten: $greeting" } + .flatten() + .view { greeting -> "After flatten: $greeting" } ``` === "Before" @@ -549,7 +549,7 @@ In the workflow block, make the following code change: ```groovy title="hello-channels.nf" linenums="31" // create a channel for inputs greeting_ch = Channel.of(greetings_array) - .flatten() + .flatten() ``` We are using an operator _closure_ here - the curly brackets. @@ -602,7 +602,7 @@ Importantly, this means each item can now be processed separately by the workflo ```groovy title="hello-channels.nf" linenums="31" // create a channel for inputs greeting_ch = Channel.of(greetings_array) - .flatten() + .flatten() ``` We left them in the `hello-channels-3.nf` solution file for reference purposes. @@ -681,7 +681,7 @@ In the workflow block, make the following code change: ```groovy title="hello-channels.nf" linenums="31" // create a channel for inputs greeting_ch = Channel.of(greetings_array) - .flatten() + .flatten() ``` #### 4.1.3. Run the workflow @@ -736,9 +736,9 @@ In the workflow block, make the following code change: ```groovy title="hello-channels.nf" linenums="31" hl_lines="3-5" // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) - .view { csv -> "Before splitCsv: $csv" } - .splitCsv() - .view { csv -> "After splitCsv: $csv" } + .view { csv -> "Before splitCsv: $csv" } + .splitCsv() + .view { csv -> "After splitCsv: $csv" } ``` === "Before" @@ -819,11 +819,11 @@ In the workflow block, make the following code change: ```groovy title="hello-channels.nf" linenums="31" hl_lines="6-8" // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) - .view { csv -> "Before splitCsv: $csv" } - .splitCsv() - .view { csv -> "After splitCsv: $csv" } - .map { item -> item[0] } - .view { csv -> "After map: $csv" } + .view { csv -> "Before splitCsv: $csv" } + .splitCsv() + .view { csv -> "After splitCsv: $csv" } + .map { item -> item[0] } + .view { csv -> "After map: $csv" } ``` === "Before" @@ -831,9 +831,9 @@ In the workflow block, make the following code change: ```groovy title="hello-channels.nf" linenums="31" // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) - .view { csv -> "Before splitCsv: $csv" } - .splitCsv() - .view { csv -> "After splitCsv: $csv" } + .view { csv -> "Before splitCsv: $csv" } + .splitCsv() + .view { csv -> "After splitCsv: $csv" } ``` From 590b7230e003493a1c5adae6de2f55a8ef8e26f2 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Sun, 26 Oct 2025 09:40:11 +0100 Subject: [PATCH 09/17] Fix starting script --- hello-nextflow/hello-world.nf | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/hello-nextflow/hello-world.nf b/hello-nextflow/hello-world.nf index c70aeae5b8..4f672da2b5 100644 --- a/hello-nextflow/hello-world.nf +++ b/hello-nextflow/hello-world.nf @@ -17,14 +17,5 @@ process sayHello { workflow { // emit a greeting - ch_output = sayHello() - - publish: - greetings = ch_output -} - -output { - greetings { - path '.' - } + sayHello() } From 953a4789b5f2097bfcb41091335f08012a449771 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Sun, 26 Oct 2025 09:51:01 +0100 Subject: [PATCH 10/17] Add main: sections to workflows with publish: blocks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Added main: sections to all workflows that have publish: blocks to comply with Nextflow linter requirements. This affects: - Documentation examples in 01_hello_world.md and 03_hello_workflow.md - Main example scripts: hello-channels.nf, hello-workflow.nf - All solution files in 1-hello-world, 2-hello-channels, 3-hello-workflow, 4-hello-modules, and 5-hello-containers directories The hello-world.nf starter file remains unchanged as it represents the initial state before students add publishing. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/hello_nextflow/01_hello_world.md | 6 ++++-- docs/hello_nextflow/03_hello_workflow.md | 4 +++- hello-nextflow/hello-channels.nf | 1 + hello-nextflow/hello-workflow.nf | 1 + hello-nextflow/solutions/1-hello-world/hello-world-3.nf | 1 + hello-nextflow/solutions/1-hello-world/hello-world-4.nf | 1 + .../solutions/2-hello-channels/hello-channels-1.nf | 1 + .../solutions/2-hello-channels/hello-channels-2.nf | 1 + .../solutions/2-hello-channels/hello-channels-3.nf | 1 + .../solutions/2-hello-channels/hello-channels-4.nf | 1 + .../solutions/3-hello-workflow/hello-workflow-1.nf | 1 + .../solutions/3-hello-workflow/hello-workflow-2.nf | 1 + .../solutions/3-hello-workflow/hello-workflow-3.nf | 1 + hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf | 1 + hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf | 1 + hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf | 1 + .../solutions/5-hello-containers/hello-containers-2.nf | 1 + 17 files changed, 22 insertions(+), 3 deletions(-) diff --git a/docs/hello_nextflow/01_hello_world.md b/docs/hello_nextflow/01_hello_world.md index 56160fd7f8..68687ddec8 100644 --- a/docs/hello_nextflow/01_hello_world.md +++ b/docs/hello_nextflow/01_hello_world.md @@ -340,6 +340,7 @@ In the workflow script file `hello-world.nf`, make the following code modificati workflow { + main: // emit a greeting ch_output = sayHello() @@ -379,8 +380,9 @@ In the workflow script file `hello-world.nf`, make the following code modificati Let's break down what we added: -1. **`publish:` section in the workflow**: This assigns the output channel from `sayHello()` to a named output called `greetings` -2. **`output` block**: This declares how the `greetings` output should be published. The `path '.'` means files will be published to the root of the output directory (which defaults to `results`) +1. **`main:` section in the workflow**: When using `publish:`, we need to explicitly mark the main workflow logic with `main:` +2. **`publish:` section in the workflow**: This assigns the output channel from `sayHello()` to a named output called `greetings` +3. **`output` block**: This declares how the `greetings` output should be published. The `path '.'` means files will be published to the root of the output directory (which defaults to `results`) #### 3.1.2. Run the workflow again diff --git a/docs/hello_nextflow/03_hello_workflow.md b/docs/hello_nextflow/03_hello_workflow.md index 7a30f0c195..10253e9389 100644 --- a/docs/hello_nextflow/03_hello_workflow.md +++ b/docs/hello_nextflow/03_hello_workflow.md @@ -247,7 +247,8 @@ In the workflow and output blocks, make the following code change: === "After" - ```groovy title="hello-workflow.nf" linenums="53" hl_lines="4 7 8 9 10 11 12 13 14 15 16 17" + ```groovy title="hello-workflow.nf" linenums="53" hl_lines="3 6 9 10 11 12 13 14 15 16 17 18 19" + main: // emit a greeting ch_hello = sayHello(greeting_ch) @@ -272,6 +273,7 @@ In the workflow and output blocks, make the following code change: === "Before" ```groovy title="hello-workflow.nf" linenums="53" + main: // emit a greeting ch_output = sayHello(greeting_ch) diff --git a/hello-nextflow/hello-channels.nf b/hello-nextflow/hello-channels.nf index 9610a01a71..35df3b8c4b 100644 --- a/hello-nextflow/hello-channels.nf +++ b/hello-nextflow/hello-channels.nf @@ -24,6 +24,7 @@ params.greeting = 'Holà mundo!' workflow { + main: // emit a greeting ch_output = sayHello(params.greeting) diff --git a/hello-nextflow/hello-workflow.nf b/hello-nextflow/hello-workflow.nf index 0660da8311..8faa480d47 100644 --- a/hello-nextflow/hello-workflow.nf +++ b/hello-nextflow/hello-workflow.nf @@ -24,6 +24,7 @@ params.greeting = 'greetings.csv' workflow { + main: // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) .splitCsv() diff --git a/hello-nextflow/solutions/1-hello-world/hello-world-3.nf b/hello-nextflow/solutions/1-hello-world/hello-world-3.nf index c70aeae5b8..70a807b424 100644 --- a/hello-nextflow/solutions/1-hello-world/hello-world-3.nf +++ b/hello-nextflow/solutions/1-hello-world/hello-world-3.nf @@ -16,6 +16,7 @@ process sayHello { workflow { + main: // emit a greeting ch_output = sayHello() diff --git a/hello-nextflow/solutions/1-hello-world/hello-world-4.nf b/hello-nextflow/solutions/1-hello-world/hello-world-4.nf index 9610a01a71..35df3b8c4b 100644 --- a/hello-nextflow/solutions/1-hello-world/hello-world-4.nf +++ b/hello-nextflow/solutions/1-hello-world/hello-world-4.nf @@ -24,6 +24,7 @@ params.greeting = 'Holà mundo!' workflow { + main: // emit a greeting ch_output = sayHello(params.greeting) diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf index 15a892e28f..f5ccbdcfe4 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf @@ -24,6 +24,7 @@ params.greeting = 'Holà mundo!' workflow { + main: // create a channel for inputs greeting_ch = Channel.of('Hello Channels!') diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf index 240544315a..e937a31d62 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf @@ -24,6 +24,7 @@ params.greeting = 'Holà mundo!' workflow { + main: // create a channel for inputs greeting_ch = Channel.of('Hello','Bonjour','Holà') diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf index b5cdfb5eba..59ae833571 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf @@ -24,6 +24,7 @@ params.greeting = 'Holà mundo' workflow { + main: greetings_array = ['Hello','Bonjour','Holà'] // create a channel for inputs diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf index c2569add8a..9ecfccbf6b 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf @@ -24,6 +24,7 @@ params.greeting = 'greetings.csv' workflow { + main: greetings_array = ['Hello','Bonjour','Holà'] // create a channel for inputs from a CSV file diff --git a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf index 82104db3d1..31aeb5bcb4 100644 --- a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf +++ b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf @@ -41,6 +41,7 @@ params.greeting = 'greetings.csv' workflow { + main: // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) .splitCsv() diff --git a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf index d28d36bb7d..81c83ea2fd 100644 --- a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf +++ b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf @@ -58,6 +58,7 @@ params.greeting = 'greetings.csv' workflow { + main: // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) .splitCsv() diff --git a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf index 3b3b9f6c1f..60511de72a 100644 --- a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf +++ b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf @@ -60,6 +60,7 @@ params.batch = 'test-batch' workflow { + main: // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) .splitCsv() diff --git a/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf b/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf index ac9eaf5e40..0212257cf7 100644 --- a/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf +++ b/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf @@ -48,6 +48,7 @@ include { sayHello } from './modules/sayHello.nf' workflow { + main: // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) .splitCsv() diff --git a/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf b/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf index c0e83e7b29..7e9cbb8263 100644 --- a/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf +++ b/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf @@ -32,6 +32,7 @@ include { convertToUpper } from './modules/convertToUpper.nf' workflow { + main: // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) .splitCsv() diff --git a/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf b/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf index cc533e4b29..845dbe93d6 100644 --- a/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf +++ b/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf @@ -13,6 +13,7 @@ include { collectGreetings } from './modules/collectGreetings.nf' workflow { + main: // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) .splitCsv() diff --git a/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf b/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf index 1a0c6d365e..3cc0a35824 100644 --- a/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf +++ b/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf @@ -15,6 +15,7 @@ include { cowpy } from './modules/cowpy.nf' workflow { + main: // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) .splitCsv() From d46a760837aeea53a21c7fbc266f29f1e9672bbf Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Sun, 26 Oct 2025 10:00:19 +0100 Subject: [PATCH 11/17] Fix highlight, move publish to note --- docs/hello_nextflow/01_hello_world.md | 44 +++++++++++++-------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/docs/hello_nextflow/01_hello_world.md b/docs/hello_nextflow/01_hello_world.md index 68687ddec8..a078a326a5 100644 --- a/docs/hello_nextflow/01_hello_world.md +++ b/docs/hello_nextflow/01_hello_world.md @@ -324,7 +324,7 @@ In the workflow script file `hello-world.nf`, make the following code modificati === "After" - ```groovy title="hello-world.nf" linenums="1" hl_lines="17-27" + ```groovy title="hello-world.nf" linenums="1" hl_lines="16-28" #!/usr/bin/env nextflow process sayHello { @@ -419,35 +419,35 @@ You can change this behavior to copy files instead using the `mode` directive in nextflow run hello-world.nf -output-dir my-results ``` -#### 3.1.3. Understanding workflow outputs vs. publishDir +!!! note "Understanding workflow outputs vs. publishDir (legacy syntax)" -Workflow outputs are the modern approach to publishing results in Nextflow. -They provide several advantages: + Workflow outputs are the modern approach to publishing results in Nextflow. + They provide several advantages: -- **Centralized**: All publishing logic is in one place rather than scattered across process definitions -- **Flexible**: You can easily control what gets published from the workflow level -- **Cleaner modules**: Processes don't need to know where their outputs should be published + - **Centralized**: All publishing logic is in one place rather than scattered across process definitions + - **Flexible**: You can easily control what gets published from the workflow level + - **Cleaner modules**: Processes don't need to know where their outputs should be published -However, you will still encounter the older `publishDir` directive when reading existing pipelines and working with nf-core modules. -The `publishDir` directive is applied at the process level and looks like this: + However, you will still encounter the older `publishDir` directive when reading existing pipelines and working with nf-core modules. + The `publishDir` directive is applied at the process level and looks like this: -```groovy -process sayHello { + ```groovy + process sayHello { - publishDir 'results', mode: 'copy' + publishDir 'results', mode: 'copy' - output: - path 'output.txt' + output: + path 'output.txt' - script: - """ - echo 'Hello World!' > output.txt - """ -} -``` + script: + """ + echo 'Hello World!' > output.txt + """ + } + ``` -The `publishDir` directive is still supported but is being phased out in favor of workflow outputs. -Understanding both approaches will help you work with both modern and legacy Nextflow code. + The `publishDir` directive is still supported but is being phased out in favor of workflow outputs. + Understanding both approaches will help you work with both modern and legacy Nextflow code. ### 3.2. Re-launch a workflow with `-resume` From 73b81a5cce8cf2c55895dbe4b625f437d16c906c Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Sun, 26 Oct 2025 10:07:32 +0100 Subject: [PATCH 12/17] Fix example --- docs/hello_nextflow/02_hello_channels.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/hello_nextflow/02_hello_channels.md b/docs/hello_nextflow/02_hello_channels.md index f8c3f9ad21..3c214efe03 100644 --- a/docs/hello_nextflow/02_hello_channels.md +++ b/docs/hello_nextflow/02_hello_channels.md @@ -120,6 +120,9 @@ In the workflow block, make the following code change: // emit a greeting ch_output = sayHello(greeting_ch) + + publish: + greetings = ch_output } ``` @@ -133,6 +136,9 @@ In the workflow block, make the following code change: // emit a greeting ch_output = sayHello(params.greeting) + + publish: + greetings = ch_output } ``` From 07a03d95aae43e044fd383b02c8d16215b27ed8b Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Sun, 26 Oct 2025 10:12:35 +0100 Subject: [PATCH 13/17] Improve publishDir explanation to educate about legacy syntax MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reframed the publishDir explanation to introduce it as legacy syntax that readers will encounter when working with existing pipelines, rather than assuming prior knowledge. The new structure: 1. Introduces publishDir as the older approach 2. Shows code example and explains what it does 3. Contrasts with modern workflow outputs approach 4. Emphasizes importance of understanding both for legacy code This helps learners understand publishDir when they encounter it in existing Nextflow pipelines and nf-core modules. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/hello_nextflow/01_hello_world.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/docs/hello_nextflow/01_hello_world.md b/docs/hello_nextflow/01_hello_world.md index a078a326a5..5a1320de60 100644 --- a/docs/hello_nextflow/01_hello_world.md +++ b/docs/hello_nextflow/01_hello_world.md @@ -421,15 +421,8 @@ You can change this behavior to copy files instead using the `mode` directive in !!! note "Understanding workflow outputs vs. publishDir (legacy syntax)" - Workflow outputs are the modern approach to publishing results in Nextflow. - They provide several advantages: - - - **Centralized**: All publishing logic is in one place rather than scattered across process definitions - - **Flexible**: You can easily control what gets published from the workflow level - - **Cleaner modules**: Processes don't need to know where their outputs should be published - - However, you will still encounter the older `publishDir` directive when reading existing pipelines and working with nf-core modules. - The `publishDir` directive is applied at the process level and looks like this: + Before workflow outputs were introduced, Nextflow used a different approach to publishing results called the `publishDir` directive. + This directive was applied at the process level and looked like this: ```groovy process sayHello { @@ -446,8 +439,17 @@ You can change this behavior to copy files instead using the `mode` directive in } ``` + The `publishDir` directive tells Nextflow to copy (or symlink) the process outputs to the specified directory. + You can set options like `mode: 'copy'` to control how files are published. + + The modern **workflow outputs** approach you just learned provides several advantages over `publishDir`: + + - **Centralized**: All publishing logic is in one place at the workflow level, rather than scattered across individual process definitions + - **Flexible**: You can easily control what gets published from the workflow level without modifying process code + - **Cleaner modules**: Processes don't need to know where their outputs should be published, making them more reusable + The `publishDir` directive is still supported but is being phased out in favor of workflow outputs. - Understanding both approaches will help you work with both modern and legacy Nextflow code. + You will encounter `publishDir` frequently when reading existing Nextflow pipelines and working with nf-core modules, so it's important to understand both approaches. ### 3.2. Re-launch a workflow with `-resume` From 60a2303b306d8bd5eb99185302a1980c20057cf1 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Mon, 27 Oct 2025 01:22:34 +0100 Subject: [PATCH 14/17] Standardize Hello Nextflow to use .out syntax without intermediate variables MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove intermediate variables (ch_output, ch_hello, ch_upper, ch_collected) across all Hello Nextflow parts (1-6) for consistency and cleaner code. Changes by part: Part 1 (Hello World): - Remove ch_output intermediate variable from solutions - Use direct .out syntax: greetings = sayHello.out - Add note about workflow.output.mode = 'copy' configuration Part 2 (Hello Channels): - Fix documentation showing incorrect ch_output usage - Add missing main: sections to code examples - Remove intermediate variables from all 4 solution files - Replace $it with explicit variable names per docs guidance - Clean up leftover greetings_array variable Part 3 (Hello Workflow): - Fix incorrect documentation explanation about intermediate variables - Remove ch_hello, ch_upper, ch_collected from all solutions - Use .out syntax: sayHello.out, convertToUpper.out, collectGreetings.out - Use named outputs: collectGreetings.out.outfile, collectGreetings.out.count - Update hello-modules.nf starter to match Part 3 final state Part 4 (Hello Modules): - Remove intermediate variables from all 3 solution files - Use consistent .out syntax throughout Part 5 (Hello Containers): - Remove intermediate variables including ch_cowpy - Use .out syntax: cowpy.out Part 6 (Hello Config): - No workflow code changes needed (config-only) Unified approach: - Processes called without assignment: sayHello(greeting_ch) - Direct .out usage in publish: processName.out - Named outputs: processName.out.name - Consistent main: and publish: structure - Modern workflow outputs (no publishDir in starters) Fixes consistency issues across 22 files (3 docs, 19 code files) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/hello_nextflow/01_hello_world.md | 12 ++-- docs/hello_nextflow/02_hello_channels.md | 24 +++++--- docs/hello_nextflow/03_hello_workflow.md | 58 +++++++++++-------- hello-nextflow/hello-channels.nf | 4 +- hello-nextflow/hello-modules.nf | 24 ++++++-- hello-nextflow/hello-workflow.nf | 8 +-- hello-nextflow/nextflow.config | 1 + .../solutions/1-hello-world/hello-world-3.nf | 4 +- .../solutions/1-hello-world/hello-world-4.nf | 4 +- .../2-hello-channels/hello-channels-1.nf | 4 +- .../2-hello-channels/hello-channels-2.nf | 4 +- .../2-hello-channels/hello-channels-3.nf | 8 +-- .../2-hello-channels/hello-channels-4.nf | 12 ++-- .../3-hello-workflow/hello-workflow-1.nf | 8 +-- .../3-hello-workflow/hello-workflow-2.nf | 16 ++--- .../3-hello-workflow/hello-workflow-3.nf | 16 ++--- .../3-hello-workflow/hello-workflow-4.nf | 19 +++--- .../4-hello-modules/hello-modules-2.nf | 14 ++--- .../4-hello-modules/hello-modules-3.nf | 14 ++--- .../4-hello-modules/hello-modules-4.nf | 14 ++--- .../5-hello-containers/hello-containers-2.nf | 18 +++--- 21 files changed, 159 insertions(+), 127 deletions(-) diff --git a/docs/hello_nextflow/01_hello_world.md b/docs/hello_nextflow/01_hello_world.md index 5a1320de60..f7aefaeeaf 100644 --- a/docs/hello_nextflow/01_hello_world.md +++ b/docs/hello_nextflow/01_hello_world.md @@ -342,10 +342,10 @@ In the workflow script file `hello-world.nf`, make the following code modificati main: // emit a greeting - ch_output = sayHello() + sayHello() publish: - greetings = ch_output + greetings = sayHello.out } output { @@ -410,7 +410,9 @@ This is how we publish results files outside of the working directories convenie By default, Nextflow creates symbolic links from the output directory to files in the work directory. This is efficient because it doesn't duplicate files, but it means if you delete the work directory, you'll lose access to the outputs. -You can change this behavior to copy files instead using the `mode` directive in the output block or by setting `workflow.output.mode = 'copy'` in your configuration file. + +In this training environment, we've configured Nextflow to copy files instead of creating symlinks, which is safer for learning purposes. +You can configure this behavior in your own pipelines using the `workflow.output.mode` setting (covered later in this training). !!! tip @@ -636,14 +638,14 @@ In the workflow block, make the following code change: ```groovy title="hello-world.nf" linenums="24" hl_lines="2" // emit a greeting - ch_output = sayHello(params.greeting) + sayHello(params.greeting) ``` === "Before" ```groovy title="hello-world.nf" linenums="24" // emit a greeting - ch_output = sayHello() + sayHello() ``` This tells Nextflow to run the `sayHello` process on the value provided through the `--greeting` parameter. diff --git a/docs/hello_nextflow/02_hello_channels.md b/docs/hello_nextflow/02_hello_channels.md index 3c214efe03..63c5417aca 100644 --- a/docs/hello_nextflow/02_hello_channels.md +++ b/docs/hello_nextflow/02_hello_channels.md @@ -81,14 +81,16 @@ In the workflow block, add the channel factory code: === "After" - ```groovy title="hello-channels.nf" linenums="27" hl_lines="3 4" + ```groovy title="hello-channels.nf" linenums="27" hl_lines="5 6" workflow { + main: + // create a channel for inputs greeting_ch = Channel.of('Hello Channels!') // emit a greeting - ch_output = sayHello(params.greeting) + sayHello(params.greeting) } ``` @@ -97,9 +99,9 @@ In the workflow block, add the channel factory code: ```groovy title="hello-channels.nf" linenums="27" workflow { + main: // emit a greeting - ch_output = sayHello(params.greeting) - } + sayHello(params.greeting) ``` This is not yet functional since we haven't yet switched the input to the process call. @@ -112,17 +114,19 @@ In the workflow block, make the following code change: === "After" - ```groovy title="hello-channels.nf" linenums="27" hl_lines="7" + ```groovy title="hello-channels.nf" linenums="27" hl_lines="9" workflow { + main: + // create a channel for inputs greeting_ch = Channel.of('Hello Channels!') // emit a greeting - ch_output = sayHello(greeting_ch) + sayHello(greeting_ch) publish: - greetings = ch_output + greetings = sayHello.out } ``` @@ -131,14 +135,16 @@ In the workflow block, make the following code change: ```groovy title="hello-channels.nf" linenums="27" workflow { + main: + // create a channel for inputs greeting_ch = Channel.of('Hello Channels!') // emit a greeting - ch_output = sayHello(params.greeting) + sayHello(params.greeting) publish: - greetings = ch_output + greetings = sayHello.out } ``` diff --git a/docs/hello_nextflow/03_hello_workflow.md b/docs/hello_nextflow/03_hello_workflow.md index 10253e9389..dfc533800c 100644 --- a/docs/hello_nextflow/03_hello_workflow.md +++ b/docs/hello_nextflow/03_hello_workflow.md @@ -140,21 +140,25 @@ In the workflow block, make the following code change: === "After" - ```groovy title="hello-workflow.nf" linenums="53" hl_lines="4 5" + ```groovy title="hello-workflow.nf" linenums="58" hl_lines="4 5" // emit a greeting sayHello(greeting_ch) // convert the greeting to uppercase convertToUpper() - } + + publish: + greetings = sayHello.out ``` === "Before" - ```groovy title="hello-workflow.nf" linenums="53" + ```groovy title="hello-workflow.nf" linenums="58" // emit a greeting sayHello(greeting_ch) - } + + publish: + greetings = sayHello.out ``` This is not yet functional because we have not specified what should be input to the `convertToUpper()` process. @@ -170,18 +174,28 @@ In the workflow block, make the following code change: === "After" - ```groovy title="hello-workflow.nf" linenums="56" hl_lines="2" + ```groovy title="hello-workflow.nf" linenums="58" hl_lines="2 5" + // emit a greeting + sayHello(greeting_ch) + // convert the greeting to uppercase convertToUpper(sayHello.out) - } + + publish: + greetings = sayHello.out ``` === "Before" - ```groovy title="hello-workflow.nf" linenums="56" + ```groovy title="hello-workflow.nf" linenums="58" hl_lines="2 5" + // emit a greeting + sayHello(greeting_ch) + // convert the greeting to uppercase convertToUpper() - } + + publish: + greetings = sayHello.out ``` For a simple case like this (one output to one input), that's all we need to do to connect two processes! @@ -247,17 +261,16 @@ In the workflow and output blocks, make the following code change: === "After" - ```groovy title="hello-workflow.nf" linenums="53" hl_lines="3 6 9 10 11 12 13 14 15 16 17 18 19" - main: + ```groovy title="hello-workflow.nf" linenums="53" hl_lines="5 16-18" // emit a greeting - ch_hello = sayHello(greeting_ch) + sayHello(greeting_ch) // convert the greeting to uppercase - ch_upper = convertToUpper(ch_hello) + convertToUpper(sayHello.out) publish: - greetings = ch_hello - uppercase = ch_upper + greetings = sayHello.out + uppercase = convertToUpper.out } output { @@ -272,13 +285,15 @@ In the workflow and output blocks, make the following code change: === "Before" - ```groovy title="hello-workflow.nf" linenums="53" - main: + ```groovy title="hello-workflow.nf" linenums="53" hl_lines=5" // emit a greeting - ch_output = sayHello(greeting_ch) + sayHello(greeting_ch) + + // convert the greeting to uppercase + convertToUpper(sayHello.out) publish: - greetings = ch_output + greetings = sayHello.out } output { @@ -290,11 +305,8 @@ In the workflow and output blocks, make the following code change: Here we made several changes: -1. **Assigned outputs to named channels**: Changed from using `sayHello.out` to `ch_hello = sayHello()` and similarly for `convertToUpper()`. This gives us named channels that we can reference in the publish section. -2. **Added both channels to the `publish:` section**: This tells Nextflow which outputs we want to publish. -3. **Declared both outputs in the `output` block**: Specifies that both should be published to the root of the output directory. - -By assigning process outputs to named channels, we can easily reference them when declaring what to publish. This is cleaner than using the `.out` notation, especially when working with multiple outputs. +1. **Added the second process output to the `publish:` section**: Added `uppercase = convertToUpper.out` to publish the uppercase greetings alongside the original greetings. +2. **Declared both outputs in the `output` block**: Added an `uppercase` section to specify that the uppercase files should also be published to the root of the output directory. Now when you run the workflow, both the original greetings and the uppercase versions will be published to the `results` directory. diff --git a/hello-nextflow/hello-channels.nf b/hello-nextflow/hello-channels.nf index 35df3b8c4b..280b7ad138 100644 --- a/hello-nextflow/hello-channels.nf +++ b/hello-nextflow/hello-channels.nf @@ -26,10 +26,10 @@ workflow { main: // emit a greeting - ch_output = sayHello(params.greeting) + sayHello(params.greeting) publish: - greetings = ch_output + greetings = sayHello.out } output { diff --git a/hello-nextflow/hello-modules.nf b/hello-nextflow/hello-modules.nf index ccf29abd63..780f8c04b7 100644 --- a/hello-nextflow/hello-modules.nf +++ b/hello-nextflow/hello-modules.nf @@ -5,8 +5,6 @@ */ process sayHello { - publishDir 'results', mode: 'copy' - input: val greeting @@ -24,8 +22,6 @@ process sayHello { */ process convertToUpper { - publishDir 'results', mode: 'copy' - input: path input_file @@ -43,8 +39,6 @@ process convertToUpper { */ process collectGreetings { - publishDir 'results', mode: 'copy' - input: path input_files val batch_name @@ -68,6 +62,7 @@ params.batch = 'test-batch' workflow { + main: // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) .splitCsv() @@ -84,4 +79,21 @@ workflow { // emit a message about the size of the batch collectGreetings.out.count.view { "There were $it greetings in this batch" } + + publish: + greetings = sayHello.out + uppercase = convertToUpper.out + collected = collectGreetings.out.outfile +} + +output { + greetings { + path '.' + } + uppercase { + path '.' + } + collected { + path '.' + } } diff --git a/hello-nextflow/hello-workflow.nf b/hello-nextflow/hello-workflow.nf index 8faa480d47..e1c2ec5793 100644 --- a/hello-nextflow/hello-workflow.nf +++ b/hello-nextflow/hello-workflow.nf @@ -27,14 +27,14 @@ workflow { main: // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) - .splitCsv() - .map { line -> line[0] } + .splitCsv() + .map { line -> line[0] } // emit a greeting - ch_output = sayHello(greeting_ch) + sayHello(greeting_ch) publish: - greetings = ch_output + greetings = sayHello.out } output { diff --git a/hello-nextflow/nextflow.config b/hello-nextflow/nextflow.config index 0a5fd46b9a..a8a6c05192 100644 --- a/hello-nextflow/nextflow.config +++ b/hello-nextflow/nextflow.config @@ -1 +1,2 @@ docker.enabled = false +workflow.output.mode = 'copy' diff --git a/hello-nextflow/solutions/1-hello-world/hello-world-3.nf b/hello-nextflow/solutions/1-hello-world/hello-world-3.nf index 70a807b424..e7a8c4a22e 100644 --- a/hello-nextflow/solutions/1-hello-world/hello-world-3.nf +++ b/hello-nextflow/solutions/1-hello-world/hello-world-3.nf @@ -18,10 +18,10 @@ workflow { main: // emit a greeting - ch_output = sayHello() + sayHello() publish: - greetings = ch_output + greetings = sayHello.out } output { diff --git a/hello-nextflow/solutions/1-hello-world/hello-world-4.nf b/hello-nextflow/solutions/1-hello-world/hello-world-4.nf index 35df3b8c4b..280b7ad138 100644 --- a/hello-nextflow/solutions/1-hello-world/hello-world-4.nf +++ b/hello-nextflow/solutions/1-hello-world/hello-world-4.nf @@ -26,10 +26,10 @@ workflow { main: // emit a greeting - ch_output = sayHello(params.greeting) + sayHello(params.greeting) publish: - greetings = ch_output + greetings = sayHello.out } output { diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf index f5ccbdcfe4..d391ed0991 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf @@ -29,10 +29,10 @@ workflow { greeting_ch = Channel.of('Hello Channels!') // emit a greeting - ch_output = sayHello(greeting_ch) + sayHello(greeting_ch) publish: - greetings = ch_output + greetings = sayHello.out } output { diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf index e937a31d62..a63d59661f 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf @@ -29,10 +29,10 @@ workflow { greeting_ch = Channel.of('Hello','Bonjour','Holà') // emit a greeting - ch_output = sayHello(greeting_ch) + sayHello(greeting_ch) publish: - greetings = ch_output + greetings = sayHello.out } output { diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf index 59ae833571..273a743a58 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf @@ -29,15 +29,15 @@ workflow { // create a channel for inputs greeting_ch = Channel.of(greetings_array) - .view { "Before flatten: $it" } + .view { greeting -> "Before flatten: $greeting" } .flatten() - .view { "After flatten: $it" } + .view { greeting -> "After flatten: $greeting" } // emit a greeting - ch_output = sayHello(greeting_ch) + sayHello(greeting_ch) publish: - greetings = ch_output + greetings = sayHello.out } output { diff --git a/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf b/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf index 9ecfccbf6b..3b4ee8af65 100644 --- a/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf +++ b/hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf @@ -25,21 +25,19 @@ params.greeting = 'greetings.csv' workflow { main: - greetings_array = ['Hello','Bonjour','Holà'] - // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) - .view { "Before splitCsv: $it" } + .view { csv -> "Before splitCsv: $csv" } .splitCsv() - .view { "After splitCsv: $it" } + .view { csv -> "After splitCsv: $csv" } .map { line -> line[0] } - .view { "After map: $it" } + .view { csv -> "After map: $csv" } // emit a greeting - ch_output = sayHello(greeting_ch) + sayHello(greeting_ch) publish: - greetings = ch_output + greetings = sayHello.out } output { diff --git a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf index 31aeb5bcb4..4161d1d345 100644 --- a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf +++ b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf @@ -48,14 +48,14 @@ workflow { .map { line -> line[0] } // emit a greeting - ch_hello = sayHello(greeting_ch) + sayHello(greeting_ch) // convert the greeting to uppercase - ch_upper = convertToUpper(ch_hello) + convertToUpper(sayHello.out) publish: - greetings = ch_hello - uppercase = ch_upper + greetings = sayHello.out + uppercase = convertToUpper.out } output { diff --git a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf index 81c83ea2fd..71f2cd0582 100644 --- a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf +++ b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf @@ -65,22 +65,22 @@ workflow { .map { line -> line[0] } // emit a greeting - ch_hello = sayHello(greeting_ch) + sayHello(greeting_ch) // convert the greeting to uppercase - ch_upper = convertToUpper(ch_hello) + convertToUpper(sayHello.out) // collect all the greetings into one file - ch_collected = collectGreetings(ch_upper.collect()) + collectGreetings(convertToUpper.out.collect()) // optional view statements - ch_upper.view { "Before collect: $it" } - ch_upper.collect().view { "After collect: $it" } + convertToUpper.out.view { "Before collect: $it" } + convertToUpper.out.collect().view { "After collect: $it" } publish: - greetings = ch_hello - uppercase = ch_upper - collected = ch_collected + greetings = sayHello.out + uppercase = convertToUpper.out + collected = collectGreetings.out } output { diff --git a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf index 60511de72a..d579264504 100644 --- a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf +++ b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf @@ -67,22 +67,22 @@ workflow { .map { line -> line[0] } // emit a greeting - ch_hello = sayHello(greeting_ch) + sayHello(greeting_ch) // convert the greeting to uppercase - ch_upper = convertToUpper(ch_hello) + convertToUpper(sayHello.out) // collect all the greetings into one file - ch_collected = collectGreetings(ch_upper.collect(), params.batch) + collectGreetings(convertToUpper.out.collect(), params.batch) // optional view statements - ch_upper.view { "Before collect: $it" } - ch_upper.collect().view { "After collect: $it" } + convertToUpper.out.view { "Before collect: $it" } + convertToUpper.out.collect().view { "After collect: $it" } publish: - greetings = ch_hello - uppercase = ch_upper - collected = ch_collected + greetings = sayHello.out + uppercase = convertToUpper.out + collected = collectGreetings.out } output { diff --git a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-4.nf b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-4.nf index 715eab412e..b97bf1126d 100644 --- a/hello-nextflow/solutions/3-hello-workflow/hello-workflow-4.nf +++ b/hello-nextflow/solutions/3-hello-workflow/hello-workflow-4.nf @@ -62,31 +62,32 @@ params.batch = 'test-batch' workflow { + main: // create a channel for inputs from a CSV file greeting_ch = Channel.fromPath(params.greeting) .splitCsv() .map { line -> line[0] } // emit a greeting - ch_hello = sayHello(greeting_ch) + sayHello(greeting_ch) // convert the greeting to uppercase - ch_upper = convertToUpper(ch_hello) + convertToUpper(sayHello.out) // collect all the greetings into one file - ch_collected = collectGreetings(ch_upper.collect(), params.batch) + collectGreetings(convertToUpper.out.collect(), params.batch) // emit a message about the size of the batch - ch_collected.count.view { "There were $it greetings in this batch" } + collectGreetings.out.count.view { "There were $it greetings in this batch" } // optional view statements - //ch_upper.view { "Before collect: $it" } - //ch_upper.collect().view { "After collect: $it" } + //convertToUpper.out.view { "Before collect: $it" } + //convertToUpper.out.collect().view { "After collect: $it" } publish: - greetings = ch_hello - uppercase = ch_upper - collected = ch_collected.outfile + greetings = sayHello.out + uppercase = convertToUpper.out + collected = collectGreetings.out.outfile } output { diff --git a/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf b/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf index 0212257cf7..7ef2976910 100644 --- a/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf +++ b/hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf @@ -55,21 +55,21 @@ workflow { .map { line -> line[0] } // emit a greeting - ch_hello = sayHello(greeting_ch) + sayHello(greeting_ch) // convert the greeting to uppercase - ch_upper = convertToUpper(ch_hello) + convertToUpper(sayHello.out) // collect all the greetings into one file - ch_collected = collectGreetings(ch_upper.collect(), params.batch) + collectGreetings(convertToUpper.out.collect(), params.batch) // emit a message about the size of the batch - ch_collected.count.view { "There were $it greetings in this batch" } + collectGreetings.out.count.view { "There were $it greetings in this batch" } publish: - greetings = ch_hello - uppercase = ch_upper - collected = ch_collected.outfile + greetings = sayHello.out + uppercase = convertToUpper.out + collected = collectGreetings.out.outfile } output { diff --git a/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf b/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf index 7e9cbb8263..bf5cb28b11 100644 --- a/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf +++ b/hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf @@ -39,21 +39,21 @@ workflow { .map { line -> line[0] } // emit a greeting - ch_hello = sayHello(greeting_ch) + sayHello(greeting_ch) // convert the greeting to uppercase - ch_upper = convertToUpper(ch_hello) + convertToUpper(sayHello.out) // collect all the greetings into one file - ch_collected = collectGreetings(ch_upper.collect(), params.batch) + collectGreetings(convertToUpper.out.collect(), params.batch) // emit a message about the size of the batch - ch_collected.count.view { "There were $it greetings in this batch" } + collectGreetings.out.count.view { "There were $it greetings in this batch" } publish: - greetings = ch_hello - uppercase = ch_upper - collected = ch_collected.outfile + greetings = sayHello.out + uppercase = convertToUpper.out + collected = collectGreetings.out.outfile } output { diff --git a/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf b/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf index 845dbe93d6..cffa5bcc8b 100644 --- a/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf +++ b/hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf @@ -20,21 +20,21 @@ workflow { .map { line -> line[0] } // emit a greeting - ch_hello = sayHello(greeting_ch) + sayHello(greeting_ch) // convert the greeting to uppercase - ch_upper = convertToUpper(ch_hello) + convertToUpper(sayHello.out) // collect all the greetings into one file - ch_collected = collectGreetings(ch_upper.collect(), params.batch) + collectGreetings(convertToUpper.out.collect(), params.batch) // emit a message about the size of the batch - ch_collected.count.view { "There were $it greetings in this batch" } + collectGreetings.out.count.view { "There were $it greetings in this batch" } publish: - greetings = ch_hello - uppercase = ch_upper - collected = ch_collected.outfile + greetings = sayHello.out + uppercase = convertToUpper.out + collected = collectGreetings.out.outfile } output { diff --git a/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf b/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf index 3cc0a35824..975249b836 100644 --- a/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf +++ b/hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf @@ -22,25 +22,25 @@ workflow { .map { line -> line[0] } // emit a greeting - ch_hello = sayHello(greeting_ch) + sayHello(greeting_ch) // convert the greeting to uppercase - ch_upper = convertToUpper(ch_hello) + convertToUpper(sayHello.out) // collect all the greetings into one file - ch_collected = collectGreetings(ch_upper.collect(), params.batch) + collectGreetings(convertToUpper.out.collect(), params.batch) // emit a message about the size of the batch - ch_collected.count.view { "There were $it greetings in this batch" } + collectGreetings.out.count.view { "There were $it greetings in this batch" } // generate ASCII art of the greetings with cowpy - ch_cowpy = cowpy(ch_collected.outfile, params.character) + cowpy(collectGreetings.out.outfile, params.character) publish: - greetings = ch_hello - uppercase = ch_upper - collected = ch_collected.outfile - ascii_art = ch_cowpy + greetings = sayHello.out + uppercase = convertToUpper.out + collected = collectGreetings.out.outfile + ascii_art = cowpy.out } output { From 72de9ffab80849527a452c4abf70c0e035ef02ae Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Mon, 27 Oct 2025 01:40:05 +0100 Subject: [PATCH 15/17] fix highlight --- docs/hello_nextflow/03_hello_workflow.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/hello_nextflow/03_hello_workflow.md b/docs/hello_nextflow/03_hello_workflow.md index dfc533800c..8054957db8 100644 --- a/docs/hello_nextflow/03_hello_workflow.md +++ b/docs/hello_nextflow/03_hello_workflow.md @@ -174,7 +174,7 @@ In the workflow block, make the following code change: === "After" - ```groovy title="hello-workflow.nf" linenums="58" hl_lines="2 5" + ```groovy title="hello-workflow.nf" linenums="58" hl_lines="5" // emit a greeting sayHello(greeting_ch) @@ -187,7 +187,7 @@ In the workflow block, make the following code change: === "Before" - ```groovy title="hello-workflow.nf" linenums="58" hl_lines="2 5" + ```groovy title="hello-workflow.nf" linenums="58" hl_lines="5" // emit a greeting sayHello(greeting_ch) @@ -261,7 +261,7 @@ In the workflow and output blocks, make the following code change: === "After" - ```groovy title="hello-workflow.nf" linenums="53" hl_lines="5 16-18" + ```groovy title="hello-workflow.nf" linenums="53" hl_lines="5 9 16-18" // emit a greeting sayHello(greeting_ch) @@ -285,7 +285,7 @@ In the workflow and output blocks, make the following code change: === "Before" - ```groovy title="hello-workflow.nf" linenums="53" hl_lines=5" + ```groovy title="hello-workflow.nf" linenums="53" hl_lines="5" // emit a greeting sayHello(greeting_ch) From 4fc6fb6b592f72c7b986b27dfd38e6f4d5db9ed8 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Mon, 27 Oct 2025 01:49:32 +0100 Subject: [PATCH 16/17] Fix Part 3 workflow blocks to show publish: sections consistently MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add missing publish: blocks to code examples in sections 2.3.1, 2.4.1, 2.4.2, 3.2.2, and 4.2 that were previously showing incomplete workflow code (ending with }). Section 4.2 changes: - Update heading: "Report the output and update publishing" - Add explanation about updating publish block with named output - Highlight line 10 (collected = collectGreetings.out.outfile) - Shows progression from publishing just greetings/uppercase to including the collected output using the named output syntax All code examples now consistently show the full workflow structure with main: and publish: sections throughout the progression. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/hello_nextflow/03_hello_workflow.md | 55 ++++++++++++++++++++---- 1 file changed, 46 insertions(+), 9 deletions(-) diff --git a/docs/hello_nextflow/03_hello_workflow.md b/docs/hello_nextflow/03_hello_workflow.md index 8054957db8..8efa51564b 100644 --- a/docs/hello_nextflow/03_hello_workflow.md +++ b/docs/hello_nextflow/03_hello_workflow.md @@ -465,7 +465,10 @@ In the workflow block, make the following code change: // collect all the greetings into one file collectGreetings(convertToUpper.out) - } + + publish: + greetings = sayHello.out + uppercase = convertToUpper.out ``` === "Before" @@ -473,7 +476,10 @@ In the workflow block, make the following code change: ```groovy title="hello-workflow.nf" linenums="75" // convert the greeting to uppercase convertToUpper(sayHello.out) - } + + publish: + greetings = sayHello.out + uppercase = convertToUpper.out ``` This connects the output of `convertToUpper()` to the input of `collectGreetings()`. @@ -532,7 +538,10 @@ In the workflow block, make the following code change: ```groovy title="hello-workflow.nf" linenums="78" hl_lines="2" // collect all the greetings into one file collectGreetings(convertToUpper.out.collect()) - } + + publish: + greetings = sayHello.out + uppercase = convertToUpper.out ``` === "Before" @@ -540,7 +549,10 @@ In the workflow block, make the following code change: ```groovy title="hello-workflow.nf" linenums="78" // collect all the greetings into one file collectGreetings(convertToUpper.out) - } + + publish: + greetings = sayHello.out + uppercase = convertToUpper.out ``` #### 2.4.2. Add some `view()` statements @@ -556,7 +568,10 @@ Let's also include a couple of `view()` statements to visualize the before and a // optional view statements convertToUpper.out.view { greeting -> "Before collect: $greeting" } convertToUpper.out.collect().view { greeting -> "After collect: $greeting" } - } + + publish: + greetings = sayHello.out + uppercase = convertToUpper.out ``` === "Before" @@ -564,7 +579,10 @@ Let's also include a couple of `view()` statements to visualize the before and a ```groovy title="hello-workflow.nf" linenums="78" // collect all the greetings into one file collectGreetings(convertToUpper.out.collect()) - } + + publish: + greetings = sayHello.out + uppercase = convertToUpper.out ``` The `view()` statements can go anywhere you want; we put them after the call for readability. @@ -736,6 +754,10 @@ In the workflow block, make the following code change: ```groovy title="hello-workflow.nf" linenums="80" hl_lines="2" // collect all the greetings into one file collectGreetings(convertToUpper.out.collect(), params.batch) + + publish: + greetings = sayHello.out + uppercase = convertToUpper.out ``` === "Before" @@ -743,6 +765,10 @@ In the workflow block, make the following code change: ```groovy title="hello-workflow.nf" linenums="80" // collect all the greetings into one file collectGreetings(convertToUpper.out.collect()) + + publish: + greetings = sayHello.out + uppercase = convertToUpper.out ``` !!! warning @@ -868,25 +894,32 @@ In the process block, make the following code change: The `emit:` tags are optional, and we could have added a tag to only one of the outputs. But as the saying goes, why not both? -### 4.2. Report the output at the end of the workflow +### 4.2. Report the output and update publishing Now that we have two outputs coming out of the `collectGreetings` process, the `collectGreetings.out` output contains two channels: - `collectGreetings.out.outfile` contains the final output file - `collectGreetings.out.count` contains the count of greetings -We could send either or both of these to another process for further work. However, in the interest of wrapping this up, we're just going to use `view()` to demonstrate that we can access and report the count of greetings. +We could send either or both of these to another process for further work. Here, we're going to use `view()` to demonstrate that we can access and report the count of greetings. + +We also need to update the `publish:` block to publish the collected output file. Since `collectGreetings` now has multiple outputs, we must use the named output `collectGreetings.out.outfile` to specify which output we want to publish. In the workflow block, make the following code change: === "After" - ```groovy title="hello-workflow.nf" linenums="82" hl_lines="4 5" + ```groovy title="hello-workflow.nf" linenums="82" hl_lines="4 5 10" // collect all the greetings into one file collectGreetings(convertToUpper.out.collect(), params.batch) // emit a message about the size of the batch collectGreetings.out.count.view { num_greetings -> "There were $num_greetings greetings in this batch" } + + publish: + greetings = sayHello.out + uppercase = convertToUpper.out + collected = collectGreetings.out.outfile ``` === "Before" @@ -894,6 +927,10 @@ In the workflow block, make the following code change: ```groovy title="hello-workflow.nf" linenums="82" // collect all the greetings into one file collectGreetings(convertToUpper.out.collect(), params.batch) + + publish: + greetings = sayHello.out + uppercase = convertToUpper.out ``` !!! note From 6012c2a68ea05f220576af1b776ac4d018fb8f6b Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Mon, 27 Oct 2025 01:52:56 +0100 Subject: [PATCH 17/17] Add workflow.output.mode coverage to Part 6 config documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Update section 0 (Warmup): - Show complete initial config file with both docker.enabled and workflow.output.mode settings - Add explanation of workflow.output.mode: 'copy' vs 'symlink' - Make connection to Part 5 (docker) and publishing covered in Part 1 Update section 1.1: - Include workflow.output.mode in both Before/After config examples - Maintains consistency as students modify config throughout tutorial This completes the loop: - Part 1: Mentions workflow.output.mode is configured (forward reference) - Part 6: Shows and explains the actual config setting 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/hello_nextflow/06_hello_config.md | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/docs/hello_nextflow/06_hello_config.md b/docs/hello_nextflow/06_hello_config.md index a7ff60e30b..6d5bad8fb9 100644 --- a/docs/hello_nextflow/06_hello_config.md +++ b/docs/hello_nextflow/06_hello_config.md @@ -27,14 +27,29 @@ By learning to utilize these configuration options effectively, you can enhance ## 0. Warmup: Check that Docker is enabled and run the Hello Config workflow -First, a quick check. There is a `nextflow.config` file in the current directory that contains the line `docker.enabled = `, where `` is either `true` or `false` depending on whether or not you've worked through Part 5 of this course in the same environment. +First, a quick check. There is a `nextflow.config` file in the current directory that should contain basic configuration settings from earlier parts of the training. -If it is set to `true`, you don't need to do anything. +You can view the current config file: -If it is set to `false`, switch it to `true` now. +```bash +cat nextflow.config +``` + +It should look something like this: + +```groovy title="nextflow.config" linenums="1" +docker.enabled = false +workflow.output.mode = 'copy' +``` + +The `docker.enabled` setting controls whether Docker containers are used (covered in Part 5). +The `workflow.output.mode` setting controls how output files are published - `'copy'` means files are copied to the output directory, while the default `'symlink'` creates symbolic links instead. -```console title="nextflow.config" linenums="1" +For this part of the training, make sure `docker.enabled` is set to `true`: + +```groovy title="nextflow.config" linenums="1" docker.enabled = true +workflow.output.mode = 'copy' ``` Once you've done that, verify that the initial workflow runs properly: @@ -90,13 +105,14 @@ To do so, we switch the value of `docker.enabled` to `false`, and add a directiv ```groovy title="nextflow.config" linenums="1" hl_lines="1-2" docker.enabled = false conda.enabled = true + workflow.output.mode = 'copy' ``` === "Before" ```groovy title="nextflow.config" linenums="1" - docker.enabled = true + workflow.output.mode = 'copy' ``` This will allow Nextflow to create and utilize Conda environments for processes that have Conda packages specified.