You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
where `vnX.Y_tZZZZ` is the `AFTER_TAG` of the latest upgrade macro and the others are paths to the relevant sources. Apps defaults to the current location. Core and Jules default to reading the `dependencies.sh` file in the Apps source. A copy of `apply_macros.py` is available at `$UMDIR/SimSys_Scripts/lfric_macros`.
176
+
177
+
.. note::
178
+
179
+
All LFRic Core tickets with macros are expected to be linked with LFRic Apps, though they may not have required an LFRic Apps development branch (although an Apps ticket should be provided). This is fine - if there is no LFRic Apps branch just checkout the LFRic Apps trunk. Then run the apply_macros script as described above and this will share the upgrade macro across both LFRic Apps and LFRic Core as needed.
180
+
169
181
.. dropdown:: New rose-stem app?
170
182
171
183
If the ticket introduces a new rose-stem app, but doesn't otherwise have a macro
@@ -191,11 +203,12 @@ then you will need to upgrade the test-suite.
191
203
rose macro --validate -M path/to/working_copy/rose-meta
metadata changes these will need to be added to the metadata
197
-
path. Please see the :ref:`rose config-edit
198
-
example<metadata_changes>`.
206
+
207
+
LFRic Apps tickets will require an LFRic Core source to use. You can do this by checking out an appropriate working copy, and exporting the environment variable `ROSE_META_PATH=/path/to/core`.
Copy file name to clipboardExpand all lines: source/WorkingPractices/inputs.rst
+19-4Lines changed: 19 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -53,7 +53,7 @@ metadata :ref:`is available <metadata_guidance>`.
53
53
work with the model.
54
54
55
55
Changes to the metadata which don't involve namelist changes may or may not
56
-
require an upgrade macro. If you are unsure whether a UM change needs an
56
+
require an upgrade macro. If you are unsure whether a change needs an
57
57
upgrade macro, then run the following command on your branch:
58
58
59
59
.. code-block::
@@ -63,10 +63,25 @@ upgrade macro, then run the following command on your branch:
63
63
If all of the tests pass then there is no requirement to add an upgrade macro.
64
64
If any of the metadata tests fail, then the developer should add a blank upgrade
65
65
macro which contains no upgrade commands but simply points the rose stem suite
66
-
to the new metadata.
66
+
to the new metadata. The SSD team are also available to advise on whether an upgrade macro is necessary.
67
67
68
-
..
69
-
The above should probably be extended to LFRic eventually.
68
+
.. tip::
69
+
70
+
When editing metadata you should always check that the new metadata appears as expected in the gui, including testing that invalid settings raise appropriate warnings. The command to open the gui is in general:
71
+
72
+
.. code-block::
73
+
74
+
rose edit -C rose-stem/app/APP-NAME
75
+
76
+
For LFRic Apps a few extra changes are required:
77
+
78
+
.. code-block::
79
+
80
+
export ROSE_PYTHONPATH=$PYTHONPATH
81
+
export ROSE_META_PATH=/path/to/valid/core
82
+
rose edit -C rose-stem/app/APP-NAME
83
+
84
+
This requires an LFRic Core working copy at an appropriate revision to be available. It is also necessary to run from the top level of the Apps working copy to ensure rose metadata paths are valid.
Copy file name to clipboardExpand all lines: source/WorkingPractices/macros.rst
+58-4Lines changed: 58 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,10 +3,11 @@
3
3
Upgrade Macros
4
4
==============
5
5
6
-
To create an upgrade macro, the developer must edit a ``versions.py`` file which is used
7
-
to update the various apps in the rose stem suite to accept the namelist changes. The upgrade
8
-
macros also form the basis of the ``rose app-upgrade`` script applied by a user wishing to
9
-
upgrade from one version of a model to the next.
6
+
.. important::
7
+
8
+
When developing Upgrade Macros, they must be tested using a test branch (see :ref:`testing`).
9
+
10
+
To create an upgrade macro, the developer must edit a ``versions.py`` file which is used to update the various apps in the rose stem suite to accept the namelist changes. The upgrade macros also form the basis of the ``rose app-upgrade`` script applied by a user wishing to upgrade from one version of a model to the next.
10
11
11
12
The ``versions.py`` file containing upgrade macros can be found in the following locations:
12
13
@@ -20,6 +21,11 @@ The ``versions.py`` file containing upgrade macros can be found in the followin
|Variations on this theme occur, eg. LFRic Apps science sections or Components in LFRic Core
28
+
23
29
24
30
Within the file a blank upgrade macro will typically look like this:
25
31
@@ -73,3 +79,51 @@ This command can then be run on a **test** branch (see :ref:`testing`).
73
79
namelists and changing the value that a particular variable takes.
74
80
A `tutorial <http://metomi.github.io/rose/doc/html/tutorial/rose/furthertopics/upgrading.html>`_
75
81
is also available.
82
+
83
+
84
+
Upgrade Macros in LFRic
85
+
-----------------------
86
+
87
+
.. warning::
88
+
89
+
Namelist files in application example directories are not currently updated by the Apply Macros script. This feature is intended to be introduced, but for now, developers still need to manually update those files.
90
+
91
+
The organisation of LFRic metadata is different from other repositories (UM + Jules) as the metadata is stored with the Science or Application section it is associated with and is then imported by other apps that require it. This helps modularise the LFRic code but complicates macro chains.
92
+
93
+
To solve this, macros in LFRic Apps are applied using a wrapper script which will read the added macros and combine them into the versions.py files for the apps where that metadata is imported. Therefore when adding macros, the macro should be added in the versions.py file in the same metadata directory as the metadata change being made. It will then be shared as appropriate by the ``apply_macros.py`` script.
94
+
95
+
For example, if a change to metadata is made in ``science/gungho/rose-meta/lfric-gungho``, the macro should be added to the ``versions.py`` file in that directory. This will then be copied to other ``versions.py`` files that import gungho metadata, eg. lfric_atm, transport etc.
96
+
97
+
It is expected that all metadata changes in LFRic Core will require change to the rose-apps in LFRic Apps, but changes to Apps must not affect Core. Therefore, the apply_macros script requires a working copy of LFRic Apps to work, but will source it's own copy of Core if required. If your only changes are to LFRic Core metadata, then you will require a linked LFRic Apps ticket and test branch, but potentially not a development branch.
98
+
99
+
.. important::
100
+
101
+
Some complex macro commands may be dependent on the order in which they are applied. As macros are copied by the wrapper script, the order they are applied will always be determined by the reverse metadata import order. For example, lfric_atm imports gungho metadata, which itself imports components/driver. If all 3 sections have an associated macro, then the macro commands would be applied in the order: components/driver, gungho, lfric_atm.
102
+
103
+
.. tip::
104
+
105
+
The wrapper script will read the ``dependencies.sh`` file in your LFRic Apps working copy and will checkout a temporary copy of the LFRic Core source if required. Some Core metadata changes will also modify the Core rose apps. In this case make sure to also commit these changes back to the core branch.
106
+
107
+
To add upgrade macros to LFRic the following steps can be followed:
108
+
109
+
110
+
111
+
1. Checkout an LFRic Apps working copy and update the core source in ``dependencies.sh`` if you have LFRic Core changes.
112
+
2. Add your upgrade macros. These **must** be added to the versions.py file in the same directory as the metadata being changed.
113
+
3. Run the Upgrade Macro script in a test branch (see :ref:`testing`). This is located in the `SimSys_Scripts github repo <https://github.com/MetOffice/SimSys_Scripts>`_ (at meto an up to date clone is available in $UMDIR/SimSys_Scripts). The syntax for running is:
114
+
115
+
.. code-block::
116
+
117
+
SimSys_Scripts/lfric_macros/apply_macros.py vnXX.Y_tTTTT -a Apps -c Core -j Jules
118
+
119
+
.. important::
120
+
121
+
**Test branches must be used for running the Apply Macros script. Do not commit the changes made by apply_macros.py to a Dev Branch**
122
+
123
+
The Apps, Core and Jules options are paths to sources for each of these. Apps will default to the present location (so it is recommended to launch from an Apps working copy). Core and Jules will default to reading the ``dependencies.sh`` file in the Apps source if not provided.
124
+
125
+
The ``vnXX.Y_tTTTT`` option must match the After Tag of your upgrade macro. When setting this, the version is the last released version of LFRic Apps. If it's a linked Apps-Core ticket, then set the ticket number as the one where the most metadata changes are being made.
126
+
127
+
.. tip::
128
+
129
+
The apply_macros script requires python >= 3.9. At the Met Office this can be achieved by ``module load scitools``.
.. todo: add an LFRic upgrade macro command... and make these more consistent?
87
+
The `apply_macros.py` script is located in the `SimSys_Scripts github repo <https://github.com/MetOffice/SimSys_Scripts>`_ (at meto an up to date clone is available in $UMDIR/SimSys_Scripts).
88
88
89
89
.. warning::
90
90
Please ensure that Cylc7 is used with `update_all.py` @vn13.5. This is fixed at HoT and either Cylc7 or Cylc8 can be used.
0 commit comments