Potential issue with "external_variables"

[sol1105]

When using CMOR to rewrite variables like areacella, volcello, ... and they have cell_measures defined that reference those variables, they are still listed under the external_variables global attribute (while being contained in the file), eg.:

dimensions:
	[...]
variables:
        [...]
	float areacella(rlat, rlon) ;
		areacella:standard_name = "cell_area" ;
		areacella:long_name = "Atmosphere Grid-Cell Area" ;
		areacella:units = "m2" ;
		areacella:cell_methods = "area: sum" ;
		areacella:cell_measures = "area: areacella" ;
		areacella:history = "2025-06-05T12:44:53Z altered by CMOR: replaced missing value flag (-9e+33) and corresponding data with standard missing value (1e+20)." ;
		areacella:missing_value = 1.e+20f ;
		areacella:_FillValue = 1.e+20f ;
		areacella:coordinates = "lat lon" ;
		areacella:grid_mapping = "crs" ;

// global attributes:
		[...]
		:external_variables = "areacella" ;
                [...]
		:cmor_version = "3.9.0" ;

This is the case with cmor 3.9.0 but also older and probably newer versions. If variables are rewritten that are referenced in the cell_measures, they should be omitted from the external_variables attribute (which would mostly but not always result in having no external_variables attribute at all).

[taylor13]

I'll try to address this issue shortly, Martin.

[durack1]

@sol1105 would it be possible to reproduce the issue above with CMOR 3.11? We have CMOR 3.12 nearing release, so if it's a persistent issue with the latest version, it'd be great to catch this before we release

[taylor13]

Before deciding how to proceed, we need to be aware of some changes in the variable metadata that should be introduced in V1.2.3 of the data request:

• In CMIP, the variables areacella, areacello, areacellg, and areacellr all represent the same thing, but on different grids.
• In CMIP7, grid information is recorded separately from the variable metadata. (It is recorded using a registered (unque) grid label.) So, it is not necessary to have 4 variables representing the same thing.
• In keeping with the branded variable approach, all of the CMIP7 cell_measures will be changed in the cmor table from things like "area: areacella" to "area: areacell" and external_variable will need to change from "areacella" to "areacell".
• The branded variable name for all 4 CMIP6 areacells will be the same: areacell_ti-hxy-u-u.
• From the user perspective (those accessing CMIP7 data), they will obtain the variable metadata from the netCDF files, including cell_measures and external_variables. Then to obtain the correct grid areas, the user will only need the branded variable name areacell_ti_hxy-u-u and the grid_id of the variable being analyzed.

Note that "volcell" can be treated similarly to areacell.

Regarding the specific issue raised above, if I understand it correctly, I agree that the new "areacell" variable should probably not have a cell_measures variable attribute or external_variables global attribute, although I'm not sure it is absolutely necessary to clean this up if it is not easy to do. Does the file pass the CF checker?

[taylor13]

and just to check ... will the grid_mapping attribute invariably be defined by CMOR? I think for most CMIP7 data it should be omitted because it is unnecessary. It should only be included when requested by the data provider.

[sol1105]

@taylor: In this example I had produced CORDEX-CMIP6 data which requires the grid_mapping attribute. It is not present by default when I write CMIP6 / 7 data.

@durack1: In above example I used the cdo cmor operator, making use of CMOR 3.9. I adapted Matts test case (https://github.com/WCRP-CMIP/cmip7-development-mip-tables) to write out areacella with CMOR 3.11 and could reproduce the issue. However, I edited the CMIP7_atmos.json CMOR table for that and set

"areacella_ti-u-hxy-u": {
    [...]
    "cell_measures": "area: areacella",

The cell_measures for areacella in that table were originally empty, which prevented the external_variables global attribute to be set.

Looking at the CMOR code, it seems that each variable referenced in the cell_measures attribute will be listed under the external_variables attribute. The test tables are of course not final, but the following construct currently shows up there: "cell_measures": "area: areacella area: areacello",, which (in a quick test) leads to external_variables: "areacella areacello".

[taylor13]

I think that it is a mistake to include in any CMOR table a cell_measures variable that self references. For example, for the variable areacella_ti-u-hxy-u', the cell_measures` should be set to the null string, not "area: areacella". In the CMIP6 tables and in the CMIP7-development tables this is done correctly, but apparently not in the CORDEX tables. We should ask CORDEX to correct their tables. Then, I think CMOR will correctly handle this case.

You mentioned there were "test tables" (are these CORDEX or CMIP test tables?) in which a cell_measuresis set to "area: areacella area: areacello". This is also wrong. There can only be a single area specification in a cell_measures. Those "test tables" should also be corrected.

Or maybe I've completely misunderstood the problem?

[sol1105]

@taylor13 Ok, so it should be correctly specified in the CMOR tables. That is certainly the easiest way to address that problem.

Regarding the CMIP7 test tables Matt is using: In Matts tables it is correctly specified as empty string currently. I however modified the cell_measures string for areacella to see if the CMOR 3.11 code still turns it into external_variables, which it does.

Regarding CMIP6 tables: It is not fully correct that this issue does not pop up in the CMIP6 cmor tables. For areacella the cell_measures is indeed an empty string, so no external_variables was set for this particular variable. But for example volcello in the Omon table looks as follows:

        "volcello": {
            [...]
            "cell_measures": "area: areacello volume: volcello", 
            [...]

and so the issue exists in CMIP6 data.

[taylor13]

Yes, volcello in CMIP5 was correctly assigned no cell_measures, but in CMIP6 the incorrect specification given above was introduced. That persists into the 1.2.2 release of the CMIP7 data request. We will want to correct that for the 1.2.3 release. Thanks for noticing this. Are there any other variables you found that have self-referencing cell_measures?

[durack1]

@sol1105 glad that we've tugged these threads a little.

The grid_mapping component of CMOR was overhauled in 3.11, with the crs variable as defined by CF now being optionally written to the file - see https://github.com/PCMDI/cmor/releases/tag/3.11.0 and #774. We have several demos of this in the https://github.com/PCMDI/DRCDP/tree/main/DataPreparationExamples/DEMO repo.

It would be worthwhile upgrading to CMOR 3.11 to test your workflows, and then seeing what if any changes will be required to get your writing consistent data.

As @taylor13 notes, there may be a need for the CORDEX-CMIP6 tables to be updated, in the case that CMOR 3.11+ is used. We have some non-backward compatible changes that are happening in CMOR 3.12, that are requiring existing repo changes to accommodate, e.g., WCRP-CMIP/CMIP6_CVs#1265 and PCMDI/DRCDP#55

[sol1105]

@taylor13:

Variables with self-referencing cell_measures seem to only exist for:
Odec.volcello
Ofx.volcello
Omon.volcello
Oyr.volcello

The double area references (cell_measures = 'area: areacella area: areacello') I found for:
Emon.albsn
Emon.albsrfc
Emon.rsdss
Emon.rsuss

[taylor13]

Thanks for pointing out these mistakes. We'll try to get them fixed soon.