Extend CLM6 PPE simulations to 2100 using anomaly forcing

[olyson]

From @linniahawkins : I would like to extend our CLM6 PPE simulations through 2100, similarly to what Daniel did for the CLM5 PPE.

[olyson]

CTSM branch info:
/glade/u/home/linnia/ctsm6_ppe/gen_ensembles/jobscripts/startup.sh

setup info for my historical sparse grid PPE cases:
/glade/u/home/linnia/ctsm6_ppe/gen_ensembles/jobscripts/setup_basecases.sh

restart:
/glade/campaign/cgd/tss/projects/PPE/ctsm6_lhc/rest/BNF_v2.n01_ctsm5.3.012_transient_lhc0000.clm2.r.2024-01-01-00000.nc

Paramfile:
/glade/work/linnia/CLM6-PPE/ctsm6_lhc/paramfiles/lhc0000.nc

[olyson]

• the cases should be two degrees
• the baseline climate should be CRU-JRA 2005-2014
• we had saved the anomalies here: /glade/campaign/cgd/tss/projects/PPE/forcing
  • you'll be able to find fullgrid 1deg anomalies in those directories
  • the "SUBSET" versions have a coastline bug (were likely not regridded)

[olyson]

I have an SSP370 case setup. It is with ctsm5.3.055, which has the anomaly forcing infrastructure we want.
This tag is more advanced from the tag LInnia used for the historical, ctsm5.3.012.
Looking through the tags between these two tags, I see that most of them are either fates, doc, or b4b-dev tags. However, a few of them implement new options in lnd_in.
One way to approach this might be to take LInnia's tag and just cherry pick the two anomaly forcing tags into that. However, those tags seem to involve some changes to externals and so I'm not sure if that will work well.
The other approach would be to adjust ctsm5.3.055 to behave like ctsm5.3.012. A list of those changes that I've identified so far:

1. Add mimics_fi to Linnia's /glade/work/linnia/CLM6-PPE/ctsm6_lhc/paramfiles/lhc0000.nc (not answer changing but the model fails when trying to read in that parameter).

2. Set snow_thermal_cond_lake_method = 'Sturm1997'. Then everything is using Sturm1997 which is what Linnea is using.

3. Set nfix_method = 'Houlton'

4. Set irrigate = .false. (although I'm not sure why it is showing up as .false. in Linnea's historical, it shows up as .true. in my SSP, need to look into this)

5. The zeroth time step is eliminated in ctsm5.3.019. That would be hard to back out but the impact is small.

6. There is an urban building energy model bug fix in ctsm5.3.024, but I don't think we have to worry about that since it only has minor effects on urban landunits.

A conservative approach would be to repeat at least part of LInnia's historical with this setup to make sure we get very similar answers (likely not bfb).

[olyson]

As pointed out by @linniahawkins , in 3. above, nfix_method should be "Bytnerowicz".
Also, snow_thermal_cond_lake should be set to "Jordan1991", as this was a bug that was fixed later after the CLM6 PPE historical was run.
Also, in 1. above, had to add leafcn_co2_base and leafcn_co2_slope to the parameter file.
Should irrigate be set to .false., as it appears to have been set in the historical? It is .true. by default in the SSP setup.

[linniahawkins]

Thanks Keith, yes I think irrigate should be set to false to match our historical simulations.

[olyson]

I've run two SSP370s with anomaly forcings, one with ctsm5.3.055 and one with BNF_v2.n01_ctsm5.3.012. The case directories are:

/glade/work/oleson/ctsm5.3.055/cime/scripts/Clm60Bgc_ctsm53055_2deg_CRUJRA2024_Control_SSP370
/glade/work/oleson/BNF_v2.n01_ctsm5.3.012/cime/scripts/Clm60Bgc_BNFv2n01ctsm53012_2deg_CRUJRA2024_Control_SSP370

They are identical in atmospheric forcing and answers are not bfb but have very small differences.
They are compared using the standard diagnostics here:
https://webext.cgd.ucar.edu/IRCP/Clm60Bgc_ctsm53055_2deg_CRUJRA2024_Control_SSP370/lnd/Clm60Bgc_ctsm53055_2deg_CRUJRA2024_Control_SSP370_2081_2100-Clm60Bgc_BNFv2n01ctsm53012_2deg_CRUJRA2024_Control_SSP370_2081_2100/setsIndex.html

I've been looking through the timeseries files I created, e.g., for Clm60Bgc_BNFv2n01ctsm53012_2deg_CRUJRA2024_Control_SSP370 here:

/glade/derecho/scratch/oleson/timeseries/Clm60Bgc_BNFv2n01ctsm53012_2deg_CRUJRA2024_Control_SSP370/lnd/proc/tseries/month_1/

There seems to be at least a couple of gridcells where the anomaly forcing doesn't seem to have been applied, e.g, as seen here for the TBOT timeseries for a single gridcell:

The repeating CRUJRA forcing is evident but I don't see any upward trend.

[linniahawkins]

Thanks Keith, I'll poke around in those directories. Two questions:

1. Are these running with the datm subset to the sparse grid meshfile?
2. Are the SSP anomalies regridded? or do we just use the nearest neighbor?

[olyson]

1. The CRUJRA datm data is subset to the sparse grid and mapalgo = 'nn'.
2. The anomaly forcing is still at 1deg and has not been regridded or subset. My impression from @djk2120 was that the subsetted version that was used for CLM5 was causing a problem for some coastal grid cells, so I've not subset the anomaly forcing for now. My notes also indicated that there wasn't much computational improvement in subsetting either: "I thought maybe we could get better computing performance by subsetting the anomaly forcing as well but the improvement is marginal (461 years/day and 5.64 pe-hrs/yr compared to 458 years/day and 5.66 pe-hrs/yr)." By the way, the BNF version of the code I ran for this SSP simulation got 465 years/day and 6.61 pe-hrs/yr.
   I'm using the default setting for the anomaly forcing, mapalgo = 'bilinear'. If we were using subsetted data then we would need to use 'nn'.
   I'm not sure what is causing the problem I noted above, maybe the anomaly forcing needs to be regridded regardless.

[olyson]

I tried 'nn' instead of 'bilinear' for the anomaly forcing mapalgo and got similar results for TBOT for the gridcell plotted above, timeseries file here:

/glade/derecho/scratch/oleson/timeseries/Clm60Bgc_BNFv2n01ctsm53012_2deg_CRUJRA2024_IrrigF-AFnn_SSP370/lnd/proc/tseries/month_1/Clm60Bgc_BNFv2n01ctsm53012_2deg_CRUJRA2024_IrrigF-AFnn_SSP370.clm2.h0.TBOT.202401-210012.nc

I regridded the 1deg anomaly forcing to 2deg, ran a new simulation, and that seemed to fix that gridcell and the other gridcell that was problematic, e.g.,:

(Also turned off irrigation). Case directory is here:

/glade/work/oleson/BNF_v2.n01_ctsm5.3.012/cime/scripts/Clm60Bgc_BNFv2n01ctsm53012_2deg_CRUJRA2024_IrrigF-AFrg_SSP370

Timeseries files are here:

/glade/derecho/scratch/oleson/timeseries/Clm60Bgc_BNFv2n01ctsm53012_2deg_CRUJRA2024_IrrigF-AFnn_SSP370/lnd/proc/tseries/month_1

Regridding of the anomaly forcing was done with:

Regrid the 1deg anomaly forcing data to 2deg.
First, generate a mapping file:
/glade/u/apps/derecho/23.09/spack/opt/spack/esmf/8.6.0/cray-mpich/8.1.27/oneapi/2023.2.1/7haa/bin/ESMF_RegridWeightGen --method conserve --source /glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/mappingdata/grids/0.9x1.25_c110307.nc --destination /glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/mappingdata/grids/1.9x2.5_c110308.nc --weight map_fv0.9x1.25_TO_1.9x2.5_conserve.250624.nc
Result is:
/glade/derecho/scratch/oleson/ANALYSIS/map_fv0.9x1.25_TO_1.9x2.5_conserve.250624.nc
Then regrid:
ncremap -t 1 -P clm --sgs_frc=landfrac --sgs_msk=landmask -m map_fv0.9x1.25_TO_1.9x2.5_conserve.250624.nc /glade/campaign/cesm/cesmdata/inputdata/atm/datm7/anomaly_forcing/CMIP6-SSP3-7.0/af.allvars.CESM.SSP3-7.0.2015-2100_c20220628.nc af.allvars.CESM.SSP3-7.0.2015-2100.1.9x2.5_c20250624.nc
Result is:
/glade/derecho/scratch/oleson/ANALYSIS/af.allvars.CESM.SSP3-7.0.2015-2100.1.9x2.5_c20250624.nc

[linniahawkins]

Great news! Thanks for sorting this out @olyson

[olyson]

Diagnostics for this new simulation (Clm60Bgc_BNFv2n01ctsm53012_2deg_CRUJRA2024_IrrigF-AFrg_SSP370) compared to the original simulation (Clm60Bgc_BNFv2n01ctsm53012_2deg_CRUJRA2024_Control_SSP370) are here.
This looks as expected to me.
But I'm happy to entertain other methods of making sure this new simulation is a good baseline for the SSP370 PPE extension.

Also, the script I used to create and run this case is here:

/glade/work/oleson/PPE/setup_run_PPE_BGC_SSP_BNF_v2.n01_ctsm5.3.012.csh

[linniahawkins]

Looks good to me, thank you Keith! I'll use this script to set up our basecases and make sure I can reproduce this.