Add support for branch run type in EAMxx #3065

bartgol · 2024-10-24T21:08:00Z

It did not require much work, basically just grab the restart case name from mct rather than XML file. I piggy backed a small cleanup of the trace gases workaround struct.

github-actions · 2024-10-24T21:09:05Z

PR Preview Action v1.4.8
🚀 Deployed preview to https://E3SM-Project.github.io/scream/pr-preview/pr-3065/
on branch `gh-pages` at 2024-10-25 20:08 UTC

bartgol · 2024-10-24T21:09:24Z

components/eamxx/src/control/atmosphere_driver.cpp

  // we have the AD add the IC file name to the GM params
  const auto& ic_pl = m_atm_params.sublist("initial_conditions");
-  if (m_case_t0<m_run_t0) {
+  if (m_run_type==RunType::Restart) {


These checks are equivalent, but I thought using the enum may be more robust if we later decide to distinguish Continue from Branch, or maybe support Hybrid (which I still don't understand).

components/eamxx/src/mct_coupling/atm_comp_mct.F90

components/eamxx/src/control/atmosphere_driver.cpp

components/eamxx/src/share/io/scream_output_manager.hpp

E3SM-Bot · 2024-10-25T15:44:05Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

Build Num: 6231
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;CIME
PULLREQUESTNUM	3065
SCREAM_SOURCE_REPO	https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA	`b621e0f`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`d61d592`
TEST_REPO_ALIAS	SCREAM

Build Information

Test Name: SCREAM_PullRequest_Autotester_Mappy

Build Num: 5971
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;CIME
PULLREQUESTNUM	3065
SCREAM_SOURCE_REPO	https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA	`b621e0f`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`d61d592`
TEST_REPO_ALIAS	SCREAM

Using Repos:

Repo: SCREAM (E3SM-Project/scream)

Pull Request Author: bartgol

E3SM-Bot · 2024-10-25T16:07:54Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

Build Num: 6231
Status: FAILED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;CIME
PULLREQUESTNUM	3065
SCREAM_SOURCE_REPO	https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA	`b621e0f`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`d61d592`
TEST_REPO_ALIAS	SCREAM

Build Information

Test Name: SCREAM_PullRequest_Autotester_Mappy

Build Num: 5971
Status: FAILED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;CIME
PULLREQUESTNUM	3065
SCREAM_SOURCE_REPO	https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA	`b621e0f`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`d61d592`
TEST_REPO_ALIAS	SCREAM

SCREAM_PullRequest_Autotester_Weaver # 6231 FAILED (click to see last 100 lines of console output)


   Use E3SM_SCREAM tag: 20241025-1551
   Use E3SM_SCREAM tag: 20241025-1551
   Use E3SM_SCREAM tag: 20241025-1551
Configure project
Configure project
Configure project
   Each . represents 1024 bytes of output
       Each . represents 1024 bytes of output
       Each . represents 1024 bytes of output
    ................................................................................ Size of output: 25K
Build project
   Each symbol represents 1024 bytes of output.
   '\''!'\'' represents an error and '\''*'\'' a warning.
    .........*.***.***.*. Size of output: 27K
Build project
   Each symbol represents 1024 bytes of output.
   '\''!'\'' represents an error and '\''*'\'' a warning.
    .. Size of output: 27K
Build project
   Each symbol represents 1024 bytes of output.
   '\''!'\'' represents an error and '\''*'\'' a warning.
    ............*.*****....****.*********.*************.*******..*****.*.*..............  Size: 49K
    ......................................................  Size: 50K
    ...................................................  Size: 99K
    .............................  Size: 99K
    ...............................................................  Size: 150K
    ..............***.............*.....................*.......*....  Size: 200K
    ...............................................  Size: 149K
    ............................  Size: 249K
    ......................*.....**..........................  Size: 49K
    ...................................  Size: 300K
    ..................................................................  Size: 199K
    .......................................................  Size: 249K
    ............................  Size: 350K
    ........................................................  Size: 299K
    ..........................................................  Size: 350K
    ...........................................................  Size: 399K
    ...............................................................  Size: 400K
    ..  Size: 450K
    ......................................................  Size: 500K
    ...........................................................................  Size: 549K
    .................................................................  Size: 599K
    ............................  Size: 449K
    .....................................................  Size: 650K
    ............................................................  Size: 499K
    ....................................  Size: 699K
    ..........................................................................  Size: 749K
    ......................................  Size: 549K
    ................................  Size: 799K
    ......................................................................................  Size: 600K
    ..........  Size: 849K
    ...........................................................  Size: 900K
    .....................................................................................  Size: 649K
    .  Size: 949K
    ..............................................................................  Size: 1000K
    ...................................................  Size: 1050K
    ..................................................................  Size: 1099K
    ............................  Size: 700K
    ...................................................................  Size: 749K
    .................  Size: 1150K
    ................................................................  Size: 1199K
    .........................................  Size: 800K
    ................................................................................  Size: 850K
    ..................................................  Size: 899K
    ...................................................  Size: 949K
    ..................................................  Size: 999K
    ...................................................  Size: 1050K
    ..................................................  Size: 1100K
    ...................................................  Size: 1149K
    ..................................................  Size: 1199K
    ..................................................  Size: 1249K
    ...................................................  Size: 1300K
    ..................................................  Size: 1349K
    ...................................................  Size: 1399K
    ........................................................  Size: 99K
    .  Size: 1249K
    .................................................  Size: 1449K
    .................................................................  Size: 1499K
    .....................................................................  Size: 1549K
    ....................................................  Size: 1599K
    ......................................................  Size: 1649K
    .......................................................  Size: 1699K
    ...............................Terminated
Starting analysis on weaver with cmd: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6231/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && export HDF5_USE_FILE_LOCKING=FALSE && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -m weaver
RUN: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6231/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && export HDF5_USE_FILE_LOCKING=FALSE && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -m weaver
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6231/scream/components/eamxx
weaver failed'
+ errors=
+ SA_FAILURES_DETAILS+=
+ [[ 1 == 0 ]]
+ [[ weaver == \m\a\p\p\y ]]
+ set +x
######################################################
FAILS DETECTED:
  SCREAM STANDALONE TESTING FAILED!
######################################################

POST BUILD TASK : SUCCESS

END OF POST BUILD TASK : 0

Finished: ABORTED

SCREAM_PullRequest_Autotester_Mappy # 5971 FAILED (click to see last 100 lines of console output)


  - maximum:
    - value: 2.82502e+08
    - indices (w/ global column index): (78,8,113)
    - lat/lon: (31.9482, 77.5623)
srun: forcing job termination

srun: Job step aborted: Waiting up to 32 seconds for job step to finish.

slurmstepd: error: *** STEP 288019.0 ON localhost CANCELLED AT 2024-10-25T10:03:02 ***

srun: error: localhost: tasks 0-1: Killed
    Start  15: io_filled_np2

141/550 Test   #7: io_basic_np2 ..........................................................***Failed    8.96 sec

srun: job 288204 queued and waiting for resources

srun: Job allocation 288204 has been revoked

srun: Force Terminated JobId=288204
    Start  19: io_packed_np2

158/549 Test #263: wind_speed ............................................................***Failed   34.53 sec

srun: job 288172 queued and waiting for resources

srun: Job allocation 288172 has been revoked
    Start 276: homme_standalone_np2_vs_np1

159/549 Test #262: surface_upward_latent_heat_flux .......................................***Failed   34.54 sec

srun: job 288171 queued and waiting for resources

srun: Job allocation 288171 has been revoked

srun: Force Terminated JobId=288171
    Start 277: homme_standalone_np3_vs_np1

160/549 Test #261: precip_surf_mass_flux .................................................***Failed   34.55 sec

srun: job 288170 queued and waiting for resources

srun: Job allocation 288170 has been revoked

srun: Force Terminated JobId=288170
    Start 278: homme_standalone_np4_vs_np1

161/549 Test #264: aodvis ................................................................***Failed   31.56 sec

srun: job 288176 queued and waiting for resources

srun: Job allocation 288176 has been revoked

srun: Force Terminated JobId=288176
    Start 279: homme_standalone_baseline_cmp

162/549 Test #265: number_paths ..........................................................***Failed   31.56 sec

srun: job 288177 queued and waiting for resources
    Start 284: p3_standalone_np2_vs_np1

163/549 Test #436: shoc_cldfrac_mam4_aci_p3_rrtmgp_np3 ...................................***Failed   29.97 sec

srun: job 288179 queued and waiting for resources

srun: Job allocation 288179 has been revoked
    Start 443: shoc_cldfrac_mam4_aci_p3_mam4_optics_rrtmgp_np3

164/549 Test #267: atm_backtend ..........................................................***Failed   28.51 sec

srun: job 288184 queued and waiting for resources
    Start 285: p3_standalone_np3_vs_np1

165/549 Test #266: aerocom_cld ...........................................................***Failed   28.52 sec

srun: job 288183 queued and waiting for resources

srun: Job allocation 288183 has been revoked
    Start 286: p3_standalone_np4_vs_np1

166/549 Test   #7: io_basic_np2 ..........................................................***Failed   25.19 sec

srun: job 288190 queued and waiting for resources

srun: Job allocation 288190 has been revoked

srun: Force Terminated JobId=288190
    Start  19: io_packed_np2

167/549 Test #268: ad_ut .................................................................***Failed   20.63 sec

srun: job 288194 queued and waiting for resources

srun: Job allocation 288194 has been revoked

srun: Force Terminated JobId=288194
    Start 287: p3_tend_check_np1

168/549 Test #269: fail ..................................................................   Passed   20.63 sec

Start 288: p3_tend_check_np2

169/549 Test #276: homme_standalone_np2_vs_np1 ...........................................   Passed    0.10 sec

Start 289: p3_tend_check_np3

170/549 Test #277: homme_standalone_np3_vs_np1 ...........................................   Passed    0.10 sec

Start 290: p3_tend_check_np4

171/549 Test #278: homme_standalone_np4_vs_np1 ...........................................   Passed    0.09 sec

Start 291: p3_standalone_baseline_cmp

172/549 Test #279: homme_standalone_baseline_cmp .........................................   Passed    0.08 sec

Start 296: shoc_standalone_np2_vs_np1

173/549 Test  #11: io_monthly_np2 ........................................................***Failed   17.99 sec

srun: job 288198 queued and waiting for resources

srun: Job allocation 288198 has been revoked
    Start  23: io_diags_np2'


errors=
SA_FAILURES_DETAILS+=
[[ 1 == 0 ]]
[[ mappy == \m\a\p\p\y ]]
[[ 0 == 1 ]]
[[ -z '' ]]
[[ 1 == 1 ]]
[[ 0 == 1 ]]
[[ 1 == 1 ]]

++ ../../cime/scripts/create_test e3sm_scream_v1_at -c -b master --wait

Terminated

POST BUILD TASK : SUCCESS

END OF POST BUILD TASK : 0

Finished: ABORTED

It's just like a restart run, but we may have a different case name

In CIME runs, use info from cpl, while in standalone use a fixed string

bartgol · 2024-10-28T22:41:52Z

@brhillman confirmed that this works as expected. Letting the AT do its course and then merging.

E3SM-Bot · 2024-10-28T23:46:48Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

Build Num: 6241
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;CIME
PULLREQUESTNUM	3065
SCREAM_SOURCE_REPO	https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA	`6fa67eb`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`7a0f81f`
TEST_REPO_ALIAS	SCREAM

Build Information

Test Name: SCREAM_PullRequest_Autotester_Mappy

Build Num: 5981
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;CIME
PULLREQUESTNUM	3065
SCREAM_SOURCE_REPO	https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA	`6fa67eb`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`7a0f81f`
TEST_REPO_ALIAS	SCREAM

Using Repos:

Repo: SCREAM (E3SM-Project/scream)

Pull Request Author: bartgol

E3SM-Bot · 2024-10-29T01:10:59Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

Build Num: 6241
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;CIME
PULLREQUESTNUM	3065
SCREAM_SOURCE_REPO	https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA	`6fa67eb`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`7a0f81f`
TEST_REPO_ALIAS	SCREAM

Build Information

Test Name: SCREAM_PullRequest_Autotester_Mappy

Build Num: 5981
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;CIME
PULLREQUESTNUM	3065
SCREAM_SOURCE_REPO	https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA	`6fa67eb`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`7a0f81f`
TEST_REPO_ALIAS	SCREAM

E3SM-Bot · 2024-10-29T01:11:16Z

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
THE LAST COMMIT TO THIS PULL REQUEST HAS NOT BEEN REVIEWED YET!

E3SM-Bot · 2024-10-29T01:11:23Z

All Jobs Finished; status = PASSED, target_sha=65dca4e0e66f238ff870783fd625aa4e7d4f5d39, However Inspection must be performed before merge can occur...

bartgol added AT: AUTOMERGE CIME Concerns infrastructure related to CIME labels Oct 24, 2024

bartgol requested a review from tcclevenger October 24, 2024 21:08

bartgol self-assigned this Oct 24, 2024

bartgol commented Oct 24, 2024

View reviewed changes

components/eamxx/src/mct_coupling/atm_comp_mct.F90 Show resolved Hide resolved

bartgol force-pushed the bartgol/eamxx/support-branch-run branch from 39afa64 to 2ca021a Compare October 24, 2024 21:12

tcclevenger reviewed Oct 24, 2024

View reviewed changes

components/eamxx/src/control/atmosphere_driver.cpp Show resolved Hide resolved

bartgol requested a review from tcclevenger October 24, 2024 21:53

tcclevenger reviewed Oct 24, 2024

View reviewed changes

components/eamxx/src/share/io/scream_output_manager.hpp Show resolved Hide resolved

bartgol force-pushed the bartgol/eamxx/support-branch-run branch 2 times, most recently from 111c78c to b621e0f Compare October 24, 2024 22:36

tcclevenger previously approved these changes Oct 25, 2024

View reviewed changes

bartgol added the AT: WIP label Oct 25, 2024

bartgol dismissed tcclevenger’s stale review via 73e3b46 October 25, 2024 20:07

bartgol force-pushed the bartgol/eamxx/support-branch-run branch from b621e0f to 73e3b46 Compare October 25, 2024 20:07

bartgol closed this Oct 25, 2024

bartgol reopened this Oct 25, 2024

bartgol added 7 commits October 25, 2024 14:22

EAMxx: enable branch run type

3422a99

It's just like a restart run, but we may have a different case name

EAMxx: simplified a bit TraceGasesWorkaround struct

25d940b

EAMxx: pass run_type to OM setup as well

82a85c8

EAMxx: fix init_timestep logic in OM

acbaa6c

EAMxx: fix compilation errors in mct interface

e1467c6

EAMxx: fix a couple of warnings

6f3afbb

EAMxx: remove namelist param for restart casename

6fa67eb

In CIME runs, use info from cpl, while in standalone use a fixed string

bartgol force-pushed the bartgol/eamxx/support-branch-run branch from 73e3b46 to 6fa67eb Compare October 25, 2024 20:22

bartgol removed the AT: WIP label Oct 28, 2024

bartgol requested a review from tcclevenger October 28, 2024 22:42

tcclevenger approved these changes Oct 29, 2024

View reviewed changes

E3SM-Bot merged commit ce3bc7f into master Oct 29, 2024
7 checks passed

E3SM-Bot removed the AT: AUTOMERGE label Oct 29, 2024

E3SM-Bot deleted the bartgol/eamxx/support-branch-run branch October 29, 2024 15:41

Add support for branch run type in EAMxx #3065

Add support for branch run type in EAMxx #3065

Uh oh!

Conversation

bartgol commented Oct 24, 2024

Uh oh!

github-actions bot commented Oct 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bartgol Oct 24, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

E3SM-Bot commented Oct 25, 2024

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

Jenkins Parameters

Build Information

Test Name: SCREAM_PullRequest_Autotester_Mappy

Jenkins Parameters

Using Repos:

Uh oh!

E3SM-Bot commented Oct 25, 2024

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

Jenkins Parameters

Build Information

Test Name: SCREAM_PullRequest_Autotester_Mappy

Jenkins Parameters

Uh oh!

bartgol commented Oct 28, 2024

Uh oh!

E3SM-Bot commented Oct 28, 2024

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

Jenkins Parameters

Build Information

Test Name: SCREAM_PullRequest_Autotester_Mappy

Jenkins Parameters

Using Repos:

Uh oh!

E3SM-Bot commented Oct 29, 2024

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

Jenkins Parameters

Build Information

Test Name: SCREAM_PullRequest_Autotester_Mappy

Jenkins Parameters

Uh oh!

E3SM-Bot commented Oct 29, 2024

Uh oh!

E3SM-Bot commented Oct 29, 2024

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 24, 2024 •

edited

Loading