Add a uw rocoto iterate action

docs/index.rst

@@ -30,24 +30,24 @@ Configuration Management
 
 The config tool suite helps you compare, transform, modify, and even validate your configuration. The package supports YAML, shell, Fortran namelist, and INI file formats. Configuration in any of these formats may use :jinja2:`Jinja2 syntax<templates>` to express values. These values can reference others, or compute new values by evaluating mathematical expressions, building paths, manipulating strings, etc.
 
-Compare Mode
-""""""""""""
+Compare Action
+""""""""""""""
 
 When the Linux diff tool just doesn't work for comparing unordered namelists with mixed-case keys, this is your go-to! The Fortran namelists are the *real* catalyst behind this gem, but it also works on the other configuration formats.
 
 | :any:`CLI documentation with examples<cli_config_compare_examples>`
 
-Realize Mode
-""""""""""""
+Realize Action
+""""""""""""""
 
-This mode renders values created by :jinja2:`Jinja2 templates<templates>`, and lets you override values in one file or object with those from others, not necessarily with the same configuration format. With ``uwtools``, you can even reference the contents of other files to build up a configuration from its pieces.
+To realize a config is to render values encoded in :jinja2:`Jinja2 expressions<templates/#expressions>`, potentially overriding values in one file or object with those from others, not necessarily with the same configuration format. With ``uwtools``, you can even reference the contents of other files to build up a configuration from its pieces.
 
 | :any:`CLI documentation with examples<cli_config_realize_examples>`
 
-Validate Mode
-"""""""""""""
+Validate Action
+"""""""""""""""
 
-In this mode, you can provide a :json-schema:`JSON Schema<>` file alongside your configuration to validate that it meets the requirements set by the schema. We've enabled robust logging to make it easier to repair your configs when problems arise.
+Provide a :json-schema:`JSON Schema<>` file alongside your configuration to validate that it meets the requirements set by the schema. We've enabled robust logging to make it easier to repair your configs when problems arise.
 
 | :any:`CLI documentation with examples<cli_config_validate_examples>`
 
@@ -57,15 +57,15 @@ Templating
 | **CLI**: ``uw template -h``
 | **API**: ``import uwtools.api.template``
 
-Render Mode
-"""""""""""
+Render Action
+"""""""""""""
 
-The ``render`` mode that gives you the full power of rendering a :jinja2:`Jinja2 template<templates>` in the same easy-to-use interface as your other workflow tools.
+This gives you the full power of rendering a :jinja2:`Jinja2 template<templates>` in the same easy-to-use interface as your other workflow tools.
 
 | :any:`CLI documentation with examples<cli_template_render_examples>`
 
-Translate Mode
-""""""""""""""
+Translate Action
+""""""""""""""""
 
 This tool helps transform legacy configuration files templated with the atparse tool (common at :noaa:`NOAA<>`) into :jinja2:`Jinja2 templates<templates>` for use with the ``uw config realize`` and ``uw template render`` tools, or their API equivalents.
 
@@ -95,17 +95,24 @@ Rocoto Configurability
 
 This tool is all about creating a configurable interface to the :rocoto:`Rocoto<>` workflow manager tool that produces the Rocoto XML for a totally arbitrary set of tasks. The ``uwtools`` package defines a structured YAML interface that relies on tasks you define to run. Paired with the uw config tool suite, this interface becomes highly configurable and requires no XML syntax!
 
-Realize Mode
-""""""""""""
+Realize Action
+""""""""""""""
 
 This is where you put in your structured YAML that defines your workflow of choice, and it pops out a verified Rocoto XML.
 
 | :any:`CLI documentation with examples<cli_rocoto_realize_examples>`
 
-Validate Mode
-"""""""""""""
+Run Action
+""""""""""
+
+Given a Rocoto XML workflow document, invoke Rocoto in a loop, monitoring its progress, until a specified task is complete.
+
+| :any:`CLI documentation with examples<cli_rocoto_run_examples>`
+
+Validate Action
+"""""""""""""""
 
-Do you already have a Rocoto XML but don't want to run Rocoto to make sure it works? Use the validate mode to check to see if Rocoto will be happy.
+Do you already have a Rocoto XML but don't want to run Rocoto to make sure it works? Use ``rocoto validate`` to check to see if Rocoto will be happy.
 
 | :any:`CLI documentation with examples<cli_rocoto_validate_examples>`
 

docs/sections/user_guide/cli/Makefile.outputs

@@ -10,6 +10,12 @@ all: $(OUTPUTS)
 $(OUTPUTS):
 	@bash $(basename $@).cmd >$@ 2>&1 | true
 
+# The following targets support semi-automated output generation: They will not run automatically,
+# but can be manually invoked (e.g. "make foo.out" given a "foo.txt" command file) to update .out
+# files. They must be invoked in a context where all commands in the .txt file are available, e.g.
+# on an HPC where the "hsi" or "rocotorun" commands are on PATH, if those are ultimately called by
+# the recipe.
+
 %.out: %.txt %.yaml
 	@bash $< >$@ 2>&1 | true
 

docs/sections/user_guide/cli/tools/execute/help.out

@@ -12,7 +12,7 @@ Required arguments:
   --classname CLASSNAME
       Name of driver class
   --task TASK
-      Driver task to execute
+      Task to execute
 
 Optional arguments:
   -h, --help

docs/sections/user_guide/cli/tools/rocoto.rst

@@ -69,6 +69,55 @@ The examples in this section use a UW YAML file ``rocoto.yaml`` with contents:
   .. literalinclude:: rocoto/realize-exec-stdout-verbose.out
      :language: xml
 
+.. _cli_rocoto_run_examples:
+
+``run``
+-------
+
+.. literalinclude:: rocoto/run-help.cmd
+   :language: text
+   :emphasize-lines: 1
+.. literalinclude:: rocoto/run-help.out
+   :language: text
+
+Examples
+^^^^^^^^
+
+.. note:: Use of ``uw rocoto run`` requires presence of the ``rocotorun`` and ``rocotostat`` executables on ``PATH``. On HPCs, this is typically achieved by loading a system module providing Rocoto.
+
+The following examples make use of this simple UW YAML for Rocoto config:
+
+.. literalinclude:: rocoto/foobar.yaml
+   :language: yaml
+
+It could be rendered to a Rocoto XML document like this:
+
+.. literalinclude:: rocoto/foobar-realize.cmd
+   :language: text
+   :emphasize-lines: 1
+.. literalinclude:: rocoto/foobar-realize.out
+   :language: xml
+
+* To run only task ``foo``, which has no dependencies:
+
+  .. literalinclude:: rocoto/run-foo.txt
+     :language: text
+     :emphasize-lines: 6
+  .. literalinclude:: rocoto/run-foo.out
+     :language: text
+
+  Note that the second invocation of ``uw rocoto run`` immediately shows task ``foo`` in its final state, without iterating the workflow.
+
+* To run task ``bar``, which depends on task ``foo``, iterating every 3 seconds:
+
+  .. literalinclude:: rocoto/run-bar.txt
+     :language: text
+     :emphasize-lines: 4
+  .. literalinclude:: rocoto/run-bar.out
+     :language: text
+
+  Note that the second invocation of ``uw rocoto run`` immediately shows task ``bar`` in its final state, without iterating the workflow.
+
 .. _cli_rocoto_validate_examples:
 
 ``validate``

docs/sections/user_guide/cli/tools/rocoto/.gitignore

@@ -1,2 +1,3 @@
 rocoto.log
 rocoto.xml
+tmp.*

docs/sections/user_guide/cli/tools/rocoto/foobar-realize.cmd

@@ -0,0 +1 @@
+RUNDIR=/some/path uw rocoto realize -c foobar.yaml

docs/sections/user_guide/cli/tools/rocoto/foobar-realize.out

@@ -0,0 +1,30 @@
+[2025-01-02T03:04:05]     INFO Schema validation succeeded for Rocoto config
+[2025-01-02T03:04:05]     INFO Schema validation succeeded for Rocoto XML
+<?xml version='1.0' encoding='utf-8'?>
+<workflow realtime="False" scheduler="slurm">
+  <cycledef group="default">202507170000 202507170000 00:00:01</cycledef>
+  <log>/some/path/log</log>
+  <task name="foo" cycledefs="default">
+    <account>wrfruc</account>
+    <cores>1</cores>
+    <partition>service</partition>
+    <queue>batch</queue>
+    <walltime>00:01:00</walltime>
+    <command>/bin/true</command>
+    <jobname>foo</jobname>
+    <join>/some/path/slurm</join>
+  </task>
+  <task name="bar" cycledefs="default">
+    <account>wrfruc</account>
+    <cores>1</cores>
+    <partition>service</partition>
+    <queue>batch</queue>
+    <walltime>00:01:00</walltime>
+    <command>/bin/true</command>
+    <jobname>bar</jobname>
+    <join>/some/path/slurm</join>
+    <dependency>
+      <taskdep task="foo"/>
+    </dependency>
+  </task>
+</workflow>

docs/sections/user_guide/cli/tools/rocoto/foobar.yaml

@@ -0,0 +1,21 @@
+common: &common
+  account: wrfruc
+  attrs: {cycledefs: default}
+  command: /bin/true
+  cores: 1
+  join: "{{ 'RUNDIR' | env }}/slurm"
+  partition: service
+  queue: batch
+  walltime: "00:01:00"
+workflow:
+  attrs: {realtime: false, scheduler: slurm}
+  cycledef:
+    - attrs: {group: default}
+      spec: 202507170000 202507170000 00:00:01
+  log: {value: "{{ 'RUNDIR' | env }}/log" }
+  tasks:
+    task_foo:
+      <<: *common
+    task_bar:
+      <<: *common
+      dependency: {taskdep: {attrs: {task: foo}}}

docs/sections/user_guide/cli/tools/rocoto/help.out

@@ -12,5 +12,7 @@ Positional arguments:
   ACTION
     realize
       Realize a Rocoto XML workflow document
+    run
+      Run a Rocoto workflow
     validate
       Validate Rocoto XML

docs/sections/user_guide/cli/tools/rocoto/run-bar.out

@@ -0,0 +1,13 @@
++ uw rocoto run --cycle 2025-07-17T00 --database db --task bar --workflow xml
+[2025-01-02T03:04:05]     INFO Iterating workflow
+[2025-01-02T03:04:05]     INFO Workflow status:
+[2025-01-02T03:04:05]     INFO CYCLE                    TASK                       JOBID               STATE         EXIT STATUS     TRIES      DURATION
+[2025-01-02T03:04:05]     INFO ================================================================================================================================
+[2025-01-02T03:04:05]     INFO 202507170000                     foo    druby://10.178.9.5:36657          SUBMITTING                   -         0           0.0
+[2025-01-02T03:04:05]     INFO 202507170000                     bar                           -                   -                   -         -             -
+[2025-01-02T03:04:05]     INFO Iterating workflow
+[2025-01-02T03:04:05]     INFO Rocoto task 'bar' for cycle 2025-07-17 00:00:00: SUBMITTING
+[2025-01-02T03:04:05]     INFO Iterating workflow
+[2025-01-02T03:04:05]     INFO Rocoto task 'bar' for cycle 2025-07-17 00:00:00: SUCCEEDED
++ uw rocoto run --cycle 2025-07-17T00 --database db --task bar --workflow xml
+[2025-01-02T03:04:05]     INFO Rocoto task 'bar' for cycle 2025-07-17 00:00:00: SUCCEEDED

docs/sections/user_guide/cli/tools/rocoto/run-bar.txt

@@ -0,0 +1,9 @@
+export RUNDIR=$(readlink -f $(mktemp -d -p $(dirname $0)))
+uw rocoto realize -c foobar.yaml >$RUNDIR/xml 2>/dev/null
+for invocation in 1 2; do (
+  cd $RUNDIR
+  set -x
+  uw rocoto run --cycle 2025-07-17T00 --database db --task bar --workflow xml
+)
+done
+rm -rf $RUNDIR

docs/sections/user_guide/cli/tools/rocoto/run-foo.out

@@ -0,0 +1,7 @@
++ uw rocoto run --cycle 2025-07-17T00 --database db --task foo --workflow xml
+[2025-01-02T03:04:05]     INFO Iterating workflow
+[2025-01-02T03:04:05]     INFO Rocoto task 'foo' for cycle 2025-07-17 00:00:00: SUBMITTING
+[2025-01-02T03:04:05]     INFO Iterating workflow
+[2025-01-02T03:04:05]     INFO Rocoto task 'foo' for cycle 2025-07-17 00:00:00: SUCCEEDED
++ uw rocoto run --cycle 2025-07-17T00 --database db --task foo --workflow xml
+[2025-01-02T03:04:05]     INFO Rocoto task 'foo' for cycle 2025-07-17 00:00:00: SUCCEEDED

docs/sections/user_guide/cli/tools/rocoto/run-foo.txt

@@ -0,0 +1,9 @@
+export RUNDIR=$(readlink -f $(mktemp -d -p $(dirname $0)))
+uw rocoto realize -c foobar.yaml >$RUNDIR/xml 2>/dev/null
+for invocation in 1 2; do (
+  cd $RUNDIR
+  set -x
+  uw rocoto run --cycle 2025-07-17T00 --database db --task foo --workflow xml
+)
+done
+rm -rf $RUNDIR

docs/sections/user_guide/cli/tools/rocoto/run-help.cmd

@@ -0,0 +1 @@
+uw rocoto run --help

docs/sections/user_guide/cli/tools/rocoto/run-help.out

@@ -0,0 +1,27 @@
+usage: uw rocoto run [--cycle CYCLE] --database DATABASE --task TASK
+                     --workflow WORKFLOW [-h] [--version] [--rate SECONDS]
+                     [--quiet] [--verbose]
+
+Run a Rocoto workflow
+
+Required arguments:
+  --cycle CYCLE
+      The cycle in ISO8601 format (e.g. yyyy-mm-ddThh)
+  --database DATABASE, -d DATABASE
+      The Rocoto database file
+  --task TASK
+      Task to execute
+  --workflow WORKFLOW, -w WORKFLOW
+      The Rocoto XML file
+
+Optional arguments:
+  -h, --help
+      Show help and exit
+  --version
+      Show version info and exit
+  --rate SECONDS, -r SECONDS
+      Delay between workflow iterations (default: 10)
+  --quiet, -q
+      Print no logging messages
+  --verbose, -v
+      Print all logging messages

src/uwtools/api/rocoto.py

@@ -6,12 +6,15 @@
 
 from typing import TYPE_CHECKING
 
-from uwtools.rocoto import realize_rocoto_xml as _realize
-from uwtools.rocoto import validate_rocoto_xml_file as _validate
+from uwtools.rocoto import DEFAULT_ITERATION_RATE
+from uwtools.rocoto import realize as _realize
+from uwtools.rocoto import run as _run
+from uwtools.rocoto import validate_file as _validate
 from uwtools.utils.api import ensure_data_source as _ensure_data_source
 from uwtools.utils.file import str2path as _str2path
 
 if TYPE_CHECKING:
+    from datetime import datetime
     from pathlib import Path
 
     from uwtools.config.formats.yaml import YAMLConfig as _YAMLConfig
@@ -40,6 +43,31 @@ def realize(
     return True
 
 
+def run(
+    cycle: datetime,
+    database: Path | str,
+    task: str,
+    workflow: Path | str,
+    rate: int = DEFAULT_ITERATION_RATE,
+) -> bool:
+    """
+    Run the specified Rocoto workflow to completion (or failure).
+
+    :param cycle: A datetime object to make available for use in the config.
+    :param database: Path to the Rocoto database file.
+    :param task: The workflow task to run.
+    :param workflow: Path to the Rocoto XML workflow document.
+    :param rate: Seconds between workflow iterations.
+    """
+    return _run(
+        cycle=cycle,
+        database=_ensure_data_source(_str2path(database), stdin_ok=False),
+        rate=rate,
+        task=task,
+        workflow=_ensure_data_source(_str2path(workflow), stdin_ok=False),
+    )
+
+
 def validate(
     xml_file: Path | str | None = None,
     stdin_ok: bool = False,