Complete CCPPization of RK microphysics #377

jimmielin · 2025-04-10T19:46:01Z

Tag name (required for release branches): sima0_06_000
Originator(s): @jimmielin

Description (include the issue title, and the keyword ['closes', 'fixes', 'resolves'] followed by the issue number):

Finish RK stratiform CCPPization
Update FCAM4 suite with RK
Add RK stratiform FPHYStest test suite

Describe any changes made to build system: N/A

Describe any changes made to the namelist: See atmos_phys updates

List any changes to the defaults for the input datasets (e.g. boundary datasets): N/A

List all files eliminated and why: N/A

List all files added and what they do:

A       cime_config/testdefs/testmods_dirs/cam/outfrq_rk_stratiform_derecho/shell_commands
A       cime_config/testdefs/testmods_dirs/cam/outfrq_rk_stratiform_derecho/user_nl_cam
   - add RK FPHYStest

List all existing files that have been modified, and describe the changes:
(Helpful git command: git diff --name-status development...<your_branch_name>)

M       cime_config/testdefs/testlist_cam.xml
   - add RK FPHYStest

M       src/data/registry.xml
   - changes to support RK

If there are new failures (compared to the test/existing-test-failures.txt file),
have them OK'd by the gatekeeper, note them here, and add them to the file.
If there are baseline differences, include the test and the reason for the
diff. What is the nature of the change? Roundoff?

derecho/intel/aux_sima:

aux_sima_intel_20250617120828: 6 tests
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM4.derecho_intel.cam-outfrq_se_cslam_analy_ic (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM4.derecho_intel.cam-outfrq_se_cslam_analy_ic NLCOMP
  - added RK to SDF.

derecho/gnu/aux_sima:

aux_sima_gnu_20250617120826: 14 tests
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_rk_stratiform_derecho (Overall: DIFF) details:
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_rk_stratiform_derecho NLCOMP
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_rk_stratiform_derecho BASELINE /glade/campaign/cesm/community/amwg/sima_baselines/latest_gnu: ERROR BFAIL baseline directory '/glade/campaign/cesm/community/amwg/sima_baselines/latest_gnu/SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_rk_stratiform_derecho' does not exist
   - new baseline.

  SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM4.derecho_gnu.cam-outfrq_se_cslam_analy_ic (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM4.derecho_gnu.cam-outfrq_se_cslam_analy_ic NLCOMP
   - added RK to SDF.

If this changes climate describe any run(s) done to evaluate the new
climate in enough detail that it(they) could be reproduced:

CAM-SIMA date used for the baseline comparison tests if different than latest:

jimmielin · 2025-05-08T17:01:50Z

All GNU regression tests passed except for newly introduced RK test. Will run full suite of tests again prior to final merging after code reviews.

aux_sima_gnu_20250508103342: 14 tests
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_rk_stratiform_derecho (Overall: DIFF) details:
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_rk_stratiform_derecho NLCOMP
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_rk_stratiform_derecho BASELINE /glade/campaign/cesm/community/amwg/sima_baselines/latest_gnu: ERROR BFAIL baseline directory '/glade/campaign/cesm/community/amwg/sima_baselines/latest_gnu/SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_rk_stratiform_derecho' does not exist
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM4.derecho_gnu.cam-outfrq_se_cslam_analy_ic (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM4.derecho_gnu.cam-outfrq_se_cslam_analy_ic NLCOMP

nusbaume

Looks good to me @jimmielin! I did have one request to add a namelist flag to the RK snapshot test, but if it doesn't work then we can just worry about it later. Thanks!

nusbaume · 2025-05-15T03:30:51Z

cime_config/testdefs/testmods_dirs/cam/outfrq_rk_stratiform_derecho/user_nl_cam

+ncdata_check = '/glade/campaign/cesm/community/amwg/sima_baselines/cam_sima_test_snapshots/cam_ne3pg3_rasch_kristjansson_rk_stratiform_snapshot_derecho_gnu_after_c20250328.nc'
+


I recently realized that when possible we should have the model exit (and thus induce a test failure) whenever the snapshot file comparison finds significant differences. I believe this can be done for this test by adding the following namelist setting to this file:

ncdata_check_err = .true.

Of course if that ends up not working for this particular test please let me know!

Thanks @nusbaume! Sadly I can't enable this option because it'll abort the model even with round-off differences. Right now the physics check data has round-off level differences against the snapshot

********** Physics Check Data Results ********** TIMESTEP: 1 Variable # Diffs Max Diff Max Diff loc (rank, col, lev) -------- ------- -------- ----------------------------- air_temperature 127 0.22E-15 (19,3,22) geopotential_height_wrt_surface 152 0.38E-15 (14,20,19) geopotential_height_wrt_surface_at_interface 150 0.42E-15 (10,4,25) tendency_of_air_temperature_due_to_model_physics 171 0.10E-14 (19,11,24) cloud_liquid_water_mixing_ratio_wrt_moist_air_and_condensed_water 2212 0.85E-15 (7,13,23) cloud_ice_mixing_ratio_wrt_moist_air_and_condensed_water 2076 0.76E-15 (7,6,24) water_vapor_mixing_ratio_wrt_moist_air_and_condensed_water 131 0.22E-15 (12,4,26) ********** End Physics Check Data Results **********

It will run for all the time slices in the snapshot to completion with comparable round-off differences but the model will abort on timestep 1 if ncdata_check_err = .true.

ERROR: Difference(s) found during ncdata check at /glade/derecho/scratch/hplin/aux_sima_gnu_20250516111902/SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_rk_stratiform_derecho.GC.aux_sima_gnu_20250516111902/bld/atm/obj/phys_init/physics_inputs.F90:1056

Hi @jimmielin, what happens if you add the following namelist option as well:

min_difference = 1e-15

It looks like by default any difference counts as a "failure", but I think with this option only differences greater than the min_difference value will count (which based off the print out should be none of them).

Of course if you don't have time for this then no worries! Just thought it might be worth it if it is easy to test. Thanks!

Thanks @nusbaume that is useful to know! Will test when Derecho is back up!

Adding my +1 for this feature

Thanks @nusbaume and @cacraigucar! I had to relax the constraint to min_difference = 2e-15 since this will trip 1e-15

tendency_of_air_temperature_due_to_model_physics 171 0.10E-14 (19,11,24)

However if I enable min_difference the log output will change; it seems like if a difference is < min_difference, it'll be ignored by the log output as well:

********** Physics Check Data Results ********** TIMESTEP: 2 No differences found! ********** End Physics Check Data Results **********

I'm not sure if we want to "see" the differences in the logs, even if they are smaller than the min_difference. I think I would prefer to since it'll help distinguish between a truly bit-for-bit vs. a ~1e-15 difference due to differences in how SIMA and CAM do things. Please let me know which way you'd prefer!

I too would like to see roundoff differences even if we allow them to "pass" the test. Perhaps we should bring this up as a discussion topic on Monday?

happy to chat about it on monday. I don't think it'd be a big lift to still include the diffs, but I lean towards keeping it the way it is because:

I'm not sure who will be looking at the test output for a PASS

I like that the current implementation allows for cleaner/clearer depictions of actual answer changes.

cacraigucar · 2025-06-13T16:25:15Z

cime_config/testdefs/testmods_dirs/cam/outfrq_rk_stratiform_derecho/user_nl_cam

+ncdata_check = '/glade/campaign/cesm/community/amwg/sima_baselines/cam_sima_test_snapshots/cam_ne3pg3_rasch_kristjansson_rk_stratiform_snapshot_derecho_gnu_after_c20250328.nc'
+


Adding my +1 for this feature

cacraigucar · 2025-06-13T17:06:14Z

cime_config/testdefs/testmods_dirs/cam/outfrq_rk_stratiform_derecho/user_nl_cam

+user_defined_sday       = 86164.10063718943
+
+! rk
+ cldfrc_dp1             =  0.10D0


I believe there are a number of these fields which should be specified via a namelist.xml file. Thoughts?

The snapshots are taken from a CAM4 aquaplanet configuration; the cldfrc scheme tuning parameter defaults in namelist_defaults_cam.xml is different for the "default" and "CAM4" configurations; the FPHYStest compset has no way of conveying this information in SIMA (tagging @nusbaume to be sure!) since the physics suite is a test SDF so it has to be overridden here:

<cldfrc_dp1 > 0.14D0 </cldfrc_dp1> <cldfrc_dp1 phys="cam5" > 0.10D0 </cldfrc_dp1> <cldfrc_dp1 phys="cam6" > 0.10D0 </cldfrc_dp1> <cldfrc_dp1 phys="cam7" > 0.10D0 </cldfrc_dp1> <cldfrc_dp1 dyn="fv" phys="cam4" > 0.10D0 </cldfrc_dp1> <cldfrc_dp1 dyn="se" phys="cam4" > 0.10D0 </cldfrc_dp1>

Perhaps we need two tests

A test which compares directly with CAM and uses these custom settings

A test which does not use the custom settings, but is used to maintain BFB testing of RK over future commits, using the values supplied by CAM-SIMA.

I believe there also needs to be a namelist.xml file, doesn't there to support the reading of RK namelist variables (and this would be needed by test 2).

I would suggest that this also would be a good discussion topic for Monday's hackathon

Thanks @cacraigucar! The namelist variables are read by the framework, they're defined here for dp1, dp2, etc:
https://github.com/ESCOMP/atmospheric_physics/blob/main/schemes/cloud_fraction/convective_cloud_cover_namelist.xml

RK namelist variables are in the prognostic_cloud_water and cloud_particle_sedimentation schemes:
https://github.com/ESCOMP/atmospheric_physics/blob/main/schemes/rasch_kristjansson/prognostic_cloud_water_namelist.xml
https://github.com/ESCOMP/atmospheric_physics/blob/main/schemes/rasch_kristjansson/cloud_particle_sedimentation_namelist.xml

Added an issue for this in the near future: #397

Intentionally not resolving this so #397 can more easily see this discussion

cacraigucar · 2025-06-13T17:08:48Z

cime_config/testdefs/testmods_dirs/cam/outfrq_rk_stratiform_derecho/user_nl_cam

+
+! qpc4
+pver = 26
+user_defined_cpwv       = 1.846e3


Why is the value specified in namelist_definition_physconst.xml being overriden? I would suggest that the baseline which gets stored from this tests depend on the settings in that structure. That way if the field gets changed, then the test will have a baseline difference indicating that this routine has been impacted. I've not checked the other values below, but I believe that all of them specified in physconst should not be listed (changed or unchanged)

The snapshots are taken from a CAM4 aquaplanet compset (QPC4) and aquaplanet simulations have a specific set of physical constants overriding the defaults. ref. CAM/bld/namelist_files/use_cases/aquaplanet_cam4.xml

 <sday >86164.10063718943</sday> <rearth >6.37100e6</rearth> <gravit >9.79764</gravit> <mwdry >28.96623324623746</mwdry> <mwh2o >18.01618112892741</mwh2o> <cpwv >1.846e3</cpwv>

Thanks @cacraigucar, I've added an issue to create this test in the near future! #397

Intentionally not resolving this so #397 can more easily see this discussion

cime_config/testdefs/testmods_dirs/cam/outfrq_rk_stratiform_derecho/user_nl_cam

src/data/registry.xml

cacraigucar · 2025-06-17T17:41:06Z

cime_config/testdefs/testmods_dirs/cam/outfrq_rk_stratiform_derecho/user_nl_cam

+user_defined_sday       = 86164.10063718943
+
+! rk
+ cldfrc_dp1             =  0.10D0


Intentionally not resolving this so #397 can more easily see this discussion

cacraigucar · 2025-06-17T17:41:13Z

cime_config/testdefs/testmods_dirs/cam/outfrq_rk_stratiform_derecho/user_nl_cam

+
+! qpc4
+pver = 26
+user_defined_cpwv       = 1.846e3


Intentionally not resolving this so #397 can more easily see this discussion

jimmielin added 2 commits March 31, 2025 17:09

Preliminary registry.xml for rk

53c67d8

Add SMS_Ln2 physics test for RK; update CAM4

eceef56

jimmielin self-assigned this Apr 10, 2025

jimmielin added this to CAM Development Apr 10, 2025

jimmielin added 2 commits May 6, 2025 15:43

Update assigned standard names

af2efc9

Merge branch 'development' into hplin/rk_stratiform

9cdfc64

jimmielin marked this pull request as ready for review May 7, 2025 20:49

jimmielin requested a review from nusbaume May 8, 2025 16:59

nusbaume approved these changes May 15, 2025

View reviewed changes

Merge branch 'development' into hplin/rk_stratiform

7473b4f

jimmielin requested a review from cacraigucar May 16, 2025 20:11

cacraigucar requested changes Jun 13, 2025

View reviewed changes

jimmielin requested a review from cacraigucar June 13, 2025 19:32

Update ncdata_check_err and tolerance

0652c91

jimmielin mentioned this pull request Jun 16, 2025

Add regression tests to generate/check baselines for individual physics schemes #397

Open

Update ncar-physics submodule

20fba9f

cacraigucar approved these changes Jun 17, 2025

View reviewed changes

jimmielin merged commit 05042d5 into ESCOMP:development Jun 17, 2025
12 checks passed

github-project-automation bot moved this to Tag in CAM Development Jun 17, 2025

		ncdata_check = '/glade/campaign/cesm/community/amwg/sima_baselines/cam_sima_test_snapshots/cam_ne3pg3_rasch_kristjansson_rk_stratiform_snapshot_derecho_gnu_after_c20250328.nc'

Complete CCPPization of RK microphysics #377

Complete CCPPization of RK microphysics #377

Uh oh!

Conversation

jimmielin commented Apr 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jimmielin commented May 8, 2025

Uh oh!

nusbaume left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peverwhee Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jimmielin commented Apr 10, 2025 •

edited

Loading

peverwhee Jun 13, 2025 •

edited

Loading