Skip to content

Commit 747b67b

Browse files
rljacobrljacob
authored
Merge branch 'mahf708/testing/gh-ci-updates' (PR #6580)
Adds eamxx tests to gh/ci suite. Also remove some tests and lower simulations lengths to make space for more specialized testing coming in the pipelines. [BFB]
2 parents e46413d + 026ff13 commit 747b67b

19 files changed

+586
-21
lines changed

‎.github/workflows/e3sm-gh-ci-cime-tests.yml‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,18 @@ on:
55
branches: [ master ]
66
paths:
77
# first, yes to these
8+
- '.github/workflows/e3sm-gh-ci-cime-tests.yml'
89
- 'cime_config/**'
910
- 'components/eam/**'
11+
- 'components/eamxx/**'
1012
- 'components/elm/**'
1113
- 'driver-moab/**'
1214
- 'driver-mct/**'
1315
# second, no to these
1416
- '!components/eam/docs/**'
1517
- '!components/eam/mkdocs.yml'
18+
- '!components/eamxx/docs/**'
19+
- '!components/eamxx/mkdocs.yml'
1620
- '!components/elm/docs/**'
1721
- '!components/elm/mkdocs.yml'
1822

@@ -26,15 +30,11 @@ jobs:
2630
strategy:
2731
fail-fast: false
2832
matrix:
29-
test:
30-
- SMS_D_P4.ne4pg2_oQU480.F2010.singularity2_gnu
31-
- SMS_P4.ne4pg2_oQU480.F2010.singularity2_gnu
32-
- REP_P4.ne4pg2_oQU480.F2010.singularity2_gnu
33-
- ERS_P4.ne4pg2_oQU480.F2010.singularity2_gnu
34-
- ERS_P4.ne4pg2_oQU480.F2010.singularity2_gnu.eam-wcprod_F2010
35-
- ERP_P4.ne4pg2_oQU480.F2010.singularity2_gnu
36-
- PET_P4.ne4pg2_oQU480.F2010.singularity2_gnu
37-
- PEM_P4.ne4pg2_oQU480.F2010.singularity2_gnu
33+
test:
34+
- SMS_D_Ln5_P4.ne4pg2_oQU480.F2010.ghci-oci_gnu
35+
- ERS_Ld5_P4.ne4pg2_oQU480.F2010.ghci-oci_gnu.eam-wcprod_F2010
36+
- SMS_D_Ln5_P4.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.ghci-oci_gnu
37+
- ERS_Ld5_P4.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.ghci-oci_gnu.eamxx-prod
3838
container:
3939
image: ghcr.io/mahf708/e3sm-imgs:v0.0.9
4040

‎.github/workflows/e3sm-gh-ci-w-cime-tests.yml‎

Lines changed: 4 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -14,19 +14,14 @@ on:
1414
jobs:
1515

1616
ci-w:
17+
if: ${{ github.event.repository.name == 'e3sm' }}
1718
runs-on: ubuntu-latest
1819
strategy:
1920
fail-fast: false
2021
matrix:
21-
test:
22-
- SMS_D_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
23-
- SMS_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
24-
- REP_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
25-
- ERS_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
26-
- ERS_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu.allactive-wcprod_1850
27-
- ERP_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
28-
- PET_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
29-
- PEM_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
22+
test:
23+
- SMS_D_Ld1_P8.ne4pg2_oQU480.WCYCL2010NS.ghci-oci_gnu
24+
- ERS_Ld3_P8.ne4pg2_oQU480.WCYCL2010NS.ghci-oci_gnu.allactive-wcprod_1850
3025
container:
3126
image: ghcr.io/mahf708/e3sm-imgs:v0.0.9
3227

‎cime_config/config_files.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -348,6 +348,7 @@
348348
<value component="cice" >$COMP_ROOT_DIR_ICE/cime_config/testdefs/testmods_dirs</value>
349349
<value component="mosart" >$COMP_ROOT_DIR_ROF/cime_config/testdefs/testmods_dirs</value>
350350
<value component="scream" >$COMP_ROOT_DIR_ATM/cime_config/testdefs/testmods_dirs</value>
351+
<value component="eamxx" >$COMP_ROOT_DIR_ATM/cime_config/testdefs/testmods_dirs</value>
351352
<value component="mpaso" >$COMP_ROOT_DIR_OCN/cime_config/testdefs/testmods_dirs</value>
352353
<value component="mpassi" >$COMP_ROOT_DIR_ICE/cime_config/testdefs/testmods_dirs</value>
353354
<value component="ww3" >$COMP_ROOT_DIR_WAV/cime_config/testdefs/testmods_dirs</value>

‎cime_config/machines/config_machines.xml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1714,9 +1714,9 @@ commented out until "*** No rule to make target '.../libadios2pio-nm-lib.a'" iss
17141714
</environment_variables>
17151715
</machine>
17161716

1717-
<machine MACH="singularity2">
1718-
<DESC>Singularity container 2.0</DESC>
1719-
<NODENAME_REGEX>singularity2</NODENAME_REGEX>
1717+
<machine MACH="ghci-oci">
1718+
<DESC>OCI-based container 2.0</DESC>
1719+
<NODENAME_REGEX>ghci-oci</NODENAME_REGEX>
17201720
<OS>LINUX</OS>
17211721
<COMPILERS>gnu</COMPILERS>
17221722
<MPILIBS>mpich</MPILIBS>

‎components/eamxx/cime_config/testdefs/testmods_dirs/eamxx/prod/shell_commands‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
#!/bin/bash
2+
3+
cime_root=$(./xmlquery --value CIMEROOT)
4+
input_data_dir=$(./xmlquery --value DIN_LOC_ROOT)
5+
atmchange=$cime_root/../components/eamxx/scripts/atmchange
6+
7+
# Change run length
8+
./xmlchange RUN_STARTDATE="1994-10-01"
9+
10+
# Turn on budget reporting
11+
./xmlchange BUDGETS=TRUE
12+
13+
# For big data
14+
./xmlchange PIO_NETCDF_FORMAT="64bit_data"
15+
./xmlchange PIO_TYPENAME=pnetcdf #adios #,PIO_TYPENAME_ATM=adios
16+
./xmlchange PIO_REARRANGER=1 # use PIO_REARRANGER=3, for ADIOS; PIO_REARRANGER=1 for pnetcdf
17+
18+
# Turn on cosp and set frequency
19+
$atmchange -b physics::atm_procs_list="mac_aero_mic,rrtmgp,cosp"
20+
$atmchange -b physics::cosp::cosp_frequency_units="hours"
21+
$atmchange -b physics::cosp::cosp_frequency=1
22+
23+
# Need to explicitly turn on computing tendencies
24+
$atmchange -b physics::mac_aero_mic::shoc::compute_tendencies=T_mid,qv
25+
$atmchange -b physics::mac_aero_mic::p3::compute_tendencies=T_mid,qv
26+
$atmchange -b physics::rrtmgp::compute_tendencies=T_mid
27+
$atmchange -b homme::compute_tendencies=T_mid,qv
28+
29+
# Set temperature cut off in dycore threshold to 180K
30+
$atmchange -b vtheta_thresh=180
31+
32+
# Change lambda_high
33+
$atmchange -b lambda_high=0.08
34+
35+
# use GHG levels more appropriate for sim
36+
# Average from 19940101 - 20150101
37+
$atmchange -b co2vmr=377.2e-6
38+
$atmchange -b ch4vmr=1786.6e-9
39+
$atmchange -b n2ovmr=318.6e-9
40+
$atmchange -b orbital_year=-9999
41+
# use CO2 the same in land model
42+
./xmlchange CCSM_CO2_PPMV=377.2
43+
44+
# determine grid and set remap files
45+
atm_grid=$(./xmlquery --value ATM_GRID)
46+
if [[ "${atm_grid}" = "ne30np4.pg2" ]]; then
47+
hmapfile="${input_data_dir}/atm/scream/maps/map_ne30pg2_to_ne4pg2_20231201.nc"
48+
armmapfile="${input_data_dir}/atm/scream/maps/map_ne30pg2_to_DecadalSites_c20240130.nc"
49+
# Run with bugfixed SPA file
50+
$atmchange -b spa_data_file="${input_data_dir}/atm/scream/init/spa_v3.LR.F2010.2011-2025.c_20240405.nc"
51+
elif [[ "${atm_grid}" = "ne4np4.pg2" ]]; then
52+
hmapfile="${input_data_dir}/atm/scream/maps/map_ne4pg2_to_ne2pg2_c20240902.nc"
53+
echo "Note: arm remap only works for ne30pg2 atm grids for now"
54+
armmapfile="not-supported-yet"
55+
# Keep default SPA file
56+
# ... (do nothing)
57+
else
58+
echo "Note: horiz/arm remaps only work for ne30pg2 and ne4pg2 atm grids for now"
59+
hmapfile="not-supported-yet"
60+
armmapfile="not-supported-yet"
61+
fi
62+
63+
# set the output yaml files
64+
output_yaml_files=$(find ${cime_root}/../components/eamxx/cime_config/testdefs/testmods_dirs/scream/v1prod/yaml_outs/ -maxdepth 1 -type f)
65+
for file in ${output_yaml_files[@]}; do
66+
# if the word "coarse" is in the file name, do nothing
67+
if [[ "${file}" == *"_coarse.yaml" && "${hmapfile}" == "not-supported-yet" ]]; then
68+
continue
69+
elif [[ "${file}" == *"_arm.yaml" && "${armmapfile}" == "not-supported-yet" ]]; then
70+
continue
71+
else
72+
# TODO: add remap file replacement for different grids
73+
cp -v ${file} ./
74+
if [ "${file}" == "${output_yaml_files[0]}" ]; then
75+
# First file, reset output list
76+
$atmchange -b output_yaml_files="./$(basename ${file})"
77+
else
78+
# Append to output list
79+
$atmchange -b output_yaml_files+="./$(basename ${file})"
80+
fi
81+
# Replace remap files
82+
sed -i "s|horiz_remap_file:.*_to_ne30.*|horiz_remap_file: ${hmapfile}|" ./$(basename ${file})
83+
sed -i "s|horiz_remap_file:.*_to_DecadalSites.*|horiz_remap_file: ${armmapfile}|" ./$(basename ${file})
84+
fi
85+
done
86+
87+
# TODO:
88+
# the only thing remaining is to set the SST data, but for some reason, this is proving difficult
89+
# because it is not super important, leave it commented out for now, until a better solution can be found
90+
# ... also, not sure if the below are specific to a given ocn/sea-ice grid?
91+
92+
# # Point to new SST forcing
93+
# ./xmlchange --file env_run.xml --id SSTICE_DATA_FILENAME --val "${input_data_dir}/atm/cam/sst/sst_ostia_3600x7200_19940930_20151231_c20240125.nc"
94+
# ./xmlchange --file env_run.xml --id SSTICE_GRID_FILENAME --val "${input_data_dir}/ocn/docn7/domain.ocn.3600x7200.230522.nc"
95+
# ./xmlchange --file env_run.xml --id SSTICE_YEAR_ALIGN --val 1994
96+
# ./xmlchange --file env_run.xml --id SSTICE_YEAR_START --val 1994
97+
# ./xmlchange --file env_run.xml --id SSTICE_YEAR_END --val 2015

‎components/eamxx/cime_config/testdefs/testmods_dirs/eamxx/prod/user_nl_cpl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
ocn_surface_flux_scheme = 2

‎components/eamxx/cime_config/testdefs/testmods_dirs/eamxx/prod/user_nl_elm‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
! Override consistency checks since we know our surface data is inconsistent
2+
check_finidat_fsurdat_consistency = .false.
3+
check_finidat_year_consistency = .false.
4+
check_finidat_pct_consistency = .false.
5+
check_dynpft_consistency = .false.
6+
7+
hist_dov2xy = .true.,.true.
8+
hist_mfilt = 1,1
9+
hist_nhtfrq = -24,-24
10+
hist_avgflag_pertape = 'A','A'
11+
hist_fincl1 = 'FIRE', 'FPSN', 'QDRAI', 'QRUNOFF', 'ZWT', 'FSAT', 'H2OSOI', 'EFLX_LH_TOT',
12+
'QVEGT', 'QVEGE', 'FSH', 'ALBD', 'ALBI', 'TBOT', 'QBOT', 'RAIN', 'SNOW',
13+
'FSDS', 'FSDSND', 'FSDSNI', 'FSDSVD', 'FSDSVI', 'FLDS'
14+
hist_fincl2 = 'H2OSNO','SOILWATER_10CM','TG'

‎components/eamxx/cime_config/testdefs/testmods_dirs/eamxx/prod/yaml_outs/scream_output.decadal.1dailyAVG_native.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
%YAML 1.1
2+
---
3+
filename_prefix: scream_output.decadal.1dailyAVG_native.h
4+
iotype: pnetcdf
5+
Averaging Type: Average
6+
Max Snapshots Per File: 1
7+
Fields:
8+
Physics PG2:
9+
Field Names:
10+
- snow_depth_land
11+
output_control:
12+
Frequency: 1
13+
frequency_units: ndays
14+
MPI Ranks in Filename: false
15+
Restart:
16+
force_new_file: true

‎components/eamxx/cime_config/testdefs/testmods_dirs/eamxx/prod/yaml_outs/scream_output.decadal.1dailyMAX_native.yaml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
%YAML 1.1
2+
---
3+
filename_prefix: scream_output.decadal.1dailyMAX_native.h
4+
iotype: pnetcdf
5+
Averaging Type: Max
6+
Max Snapshots Per File: 1
7+
Fields:
8+
Physics PG2:
9+
Field Names:
10+
- T_2m
11+
- precip_total_surf_mass_flux
12+
output_control:
13+
Frequency: 1
14+
frequency_units: ndays
15+
MPI Ranks in Filename: false
16+
Restart:
17+
force_new_file: true

‎components/eamxx/cime_config/testdefs/testmods_dirs/eamxx/prod/yaml_outs/scream_output.decadal.1dailyMIN_native.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
%YAML 1.1
2+
---
3+
filename_prefix: scream_output.decadal.1dailyMIN_native.h
4+
iotype: pnetcdf
5+
Averaging Type: Min
6+
Max Snapshots Per File: 1
7+
Fields:
8+
Physics PG2:
9+
Field Names:
10+
- T_2m
11+
output_control:
12+
Frequency: 1
13+
frequency_units: ndays
14+
MPI Ranks in Filename: false
15+
Restart:
16+
force_new_file: true

0 commit comments

Comments
 (0)