testmodifier overdefined

[jedwards4b]

@kdraeder has pointed out that the test modifier _C is overdefined in cime. In test_scheduler.py at line 681. C# is used to specify the number of instances in a multi-coupler ensemble case. But then at line 884 it is used to specify that a Gregorian calendar should be used.

I suspect that we don't actually use the _CG in any tests or this problem would have presented itself some time ago. We can allow for this by changing line 681 to be
elif case_opt.startswith("C") and not case_opt.startswith("CG"):

[jedwards4b]

Furthermore an ERI test using _CG is currently broken since the second run is set to start 2 years after the initial run and if the initial run happens on leapday the second run date will not exist.

[jedwards4b]

After further consideration I think that this is a non-issue. The ERI test is not designed to work with a transient compset or with a Gregorian calendar. I think that we should add something to confirm nether of these settings are used in a ERI test.

[billsacks]

Thinking about this a bit... it's been a while since I've looked closely at the ERI test, but it's not clear to me why it shouldn't work with a transient compset. @jedwards4b can you elaborate on why you think it shouldn't? My recollection (quite possibly wrong!) is that the ERI does 4 runs:

1. Initial run
2. Hybrid run starting from either the middle or end of (1)
3. Branch run starting from somewhere in the middle of (2)
4. Continue run starting from somewhere in the middle of (3)

and then it compares results from the end of (2), (3) and (4). I could see how things wouldn't work for a transient case if comparisons included (1), but my recollection is that they don't. I'm thinking that both branch and continue runs should be bit-for-bit in a transient case. @jedwards4b - if you explain why you think an ERI test won't work in a transient case, I'd be happy to give it more thought myself.

(From a quick look through test lists, I do see that we don't seem to have any transient ERI tests, so that does make me question this....)

[jedwards4b]

The branch run includes an offset of 2 years from the initial run: https://github.com/ESMCI/cime/blob/master/CIME/SystemTests/eri.py#L93 In a transient run this will require a discontinuity in boundary conditions. I suppose we could modify the test to only do this offset in the case of a spin-up run.

…

On Tue, Jul 1, 2025 at 11:52 AM Bill Sacks ***@***.***> wrote: *billsacks* left a comment (ESMCI/cime#4811) <#4811 (comment)> Thinking about this a bit... it's been a while since I've looked closely at the ERI test, but it's not clear to me why it shouldn't work with a transient compset. @jedwards4b <https://github.com/jedwards4b> can you elaborate on why you think it shouldn't? My recollection (quite possibly wrong!) is that the ERI does 4 runs: 1. Initial run 2. Hybrid run starting from either the middle or end of (1) 3. Branch run starting from somewhere in the middle of (2) 4. Continue run starting from somewhere in the middle of (3) and then it compares results from the end of (2), (3) and (4). I could see how things wouldn't work for a transient case if comparisons included (1), but my recollection is that they don't. I'm thinking that both branch and continue runs should be bit-for-bit in a transient case. @jedwards4b <https://github.com/jedwards4b> - if you explain why you think an ERI test won't work in a transient case, I'd be happy to give it more thought myself. (From a quick look through test lists, I do see that we don't seem to have any transient ERI tests, so that does make me question this....) — Reply to this email directly, view it on GitHub <#4811 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABOXUGDHGBVBSIQAEGN2ITT3GKVCBAVCNFSM6AAAAACAEE5QL2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMRUGYYDCNBRGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Jim Edwards STAND UP FOR SCIENCE CESM Software Engineer National Center for Atmospheric Research Boulder, CO

[billsacks]

I think that's the setting for the hybrid case (ref2), not the branch case. I think the branch (main case, without a "ref") continues continuously from the hybrid case, and I think that's what's important for the comparison: I think that the hybrid vs initial offset of 2 years should just be thought of as getting the hybrid case off the ground, after which point the initial run is irrelevant because (I think) it isn't included in comparisons.

In fact, if there were a change of years between two cases that were compared, I'm thinking that most ERI tests would fail because things like stream dates (for data models and other forcings) would differ between the cases.

[jedwards4b]

In a spinup case (eg 1850) changing the year won't matter because the boundary conditions are the same regardless of the start year. And that's my point - I think that
the ERI is designed only to work with spinup cases.

[fischer-ncar]

I did a quick look through past test results. I didn't find any transient ERI tests for cesm1 or cesm2.

[billsacks]

In a spinup case (eg 1850) changing the year won't matter because the boundary conditions are the same regardless of the start year.

Maybe it depends on the details of the case / compset. For example, in ERI_Ld9.f10_f10_mg37.I2000Clm50BgcCru.derecho_gnu.clm-default (which is in a standard test list and passes), the atmospheric forcing data differ between the ref1 case and the ref2 and main case: The ref1 case uses year-2000 atm forcings, but the ref2 and main case use year-2002 atm forcings. So the year+2 does matter in a non-transient case, it's just that it doesn't impact the results because of what is vs. isn't compared in an ERI test.

I'm trying a transient version of that test now: /glade/derecho/scratch/sacks/tests_0701-134636de/ERI_Ld9.f10_f10_mg37.IHistClm60Bgc.derecho_gnu.clm-default.0701-134636de. It's failing due to a sanity check in CTSM, but I'll try it later with a bypass of that condition.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[kdraeder]

This issue evolved from the overdefinition of _C to @jedwards4b 's question about whether the ERI test can or should handle transient cases.
@billsacks and @ekluzek developed a solution to enable ERI to handle transients in CTSM (ESCOMP #3299). I'm hoping that it can be extended to the rest of CESM soon.
That will provide the context for the right fix to _C.