Skip to content

Conversation

@bbakernoaa
Copy link
Contributor

@bbakernoaa bbakernoaa commented May 20, 2025

Description

Resolves #3712
Resolves #3827
Resolves #4248

This pull request introduces significant updates to the aerosol configuration and emission handling in the UFS-Aerosols system, streamlining file management, improving modularity, and consolidating configuration files. Key changes include restructuring aerosol configuration files, enhancing biomass burning emissions handling, and updating the chemical input file linking process.

Aerosol Configuration Updates:

  • Refactored config.aero.j2 to organize settings into distinct sections for aerosol and emission configurations, including the addition of new variables like AERO_EMIS_FIRE_VERSION and AERO_EMIS_FIRE_HIST. [1] [2]

Emission Data Management:

  • Consolidated multiple emission configuration files (ExtData.gbbepx, ExtData.qfed, and ExtData.none) into a single file, AERO_ExtData.rc, for better modularity and maintainability. Removed redundant and unused emission entries. [1] [2] [3] [4]

Chemical Input File Handling:

  • Added linking of chemical input files (CHEM_INPUT) in forecast_predet.sh to ensure proper file availability during runtime.

Removal of Legacy Code:

  • Removed the automated generation of AERO_ExtData.rc in parsing_namelists_GOCART.sh, as the new consolidated configuration file eliminates the need for dynamic generation.

Additional Updates:

  • Updated config.com to include new template paths for chemical restart and input files, improving file organization.

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO
    • EMC verif-global
    • GDAS
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

  • GCAFS noDA cycled on URSA

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

@CoryMartin-NOAA CoryMartin-NOAA requested a review from Copilot May 20, 2025 20:40
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request updates aerosol configuration and emissions handling for the UFS-Aerosols system by consolidating file management and enhancing modularity. Key changes include:

  • Removing legacy logic for generating the AERO_ExtData.rc file and consolidating emission configuration files.
  • Adding a link to chemical input files in forecast_predet.sh and updating configuration templates in both aerosol and chemical configuration files.
  • Refactoring configuration files to improve clarity and maintainability.

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
ush/parsing_namelists_GOCART.sh Removed legacy dynamic generation of AERO_ExtData.rc.
ush/forecast_predet.sh Added linking for CHEM_INPUT to ensure chemical input file availability.
parm/ufs/gocart/ExtData.* Removed obsolete emission configuration files, consolidating them into AERO_ExtData.rc.
parm/ufs/gocart/AERO_ExtData.rc Updated emission entries to use chemical input paths and adjusted emission factors.
parm/chem/chem_emission.yaml.j2 Updated templating loops for QFED variables and file copying operations.
dev/parm/config/gfs/config.com Added new chemical template paths.
dev/parm/config/gcafs/config.aero.j2 Reorganized aerosol sections and introduced new aerosol emission settings.

@bbakernoaa bbakernoaa force-pushed the feature/prep_emis_rework branch from 2e06fb3 to b534a8b Compare July 31, 2025 19:47
@bbakernoaa bbakernoaa marked this pull request as ready for review August 7, 2025 15:21
@bbakernoaa bbakernoaa requested a review from aerorahul as a code owner August 7, 2025 15:21
@bbakernoaa bbakernoaa mentioned this pull request Nov 13, 2025
59 tasks
@DavidHuber-NOAA DavidHuber-NOAA removed the CI-Gaeac6-Failed **Bot use only** CI testing on Gaea C6 for this PR has failed label Nov 17, 2025
@DavidHuber-NOAA
Copy link
Contributor

@bbakernoaa I have updated the description of this PR to close #4248 once testing is complete.

@bbakernoaa
Copy link
Contributor Author

@bbakernoaa I have updated the description of this PR to close #4248 once testing is complete.

Thanks now I need to update this workflow branch to point to this new fix directory right?

@DavidHuber-NOAA
Copy link
Contributor

@bbakernoaa Yes, that's correct.

@bbakernoaa
Copy link
Contributor Author

@DavidHuber-NOAA this should be ready for the ci tests now.

@DavidHuber-NOAA
Copy link
Contributor

Launching on C6 and WCOSS2.

@DavidHuber-NOAA DavidHuber-NOAA added the CI-Wcoss2-Ready PR is ready for CI testing on WCOSS2. label Nov 25, 2025
@DavidHuber-NOAA
Copy link
Contributor

DavidHuber-NOAA commented Nov 25, 2025

It looks like C6 is not triggering automatically, so I will run it manually.

@DavidHuber-NOAA It looks like you entered the PR number incorrectly:

  echo "Selected machines: ${MACHINES}"
  echo "MACHINES=${MACHINES}" >> "${GITHUB_ENV}"
  shell: /usr/bin/bash -e {0}
  env:
    GH_TOKEN: ***
PR 37133 not found
Error: Process completed with exit code 1.

But will still fail on build on C6 until we update GitHub CLI for checkout PRs

@DavidHuber-NOAA DavidHuber-NOAA added CI-GaeaC6-Running (CM) CI testing is being run locally on Gaea C6. CI-Wcoss2-Running CI testing on WCOSS for this PR is in-progress and removed CI-Wcoss2-Ready PR is ready for CI testing on WCOSS2. labels Nov 25, 2025
@DavidHuber-NOAA
Copy link
Contributor

@bbakernoaa Can you resolve the conflict in gcafs_tasks.py? The tests have finished building but have not started running on C6 and Cactus, so I can still incorporate this update.

@DavidHuber-NOAA
Copy link
Contributor

The tests are now running on Cactus and C6.

@DavidHuber-NOAA
Copy link
Contributor

DavidHuber-NOAA commented Nov 25, 2025

@bbakernoaa Tests are failing on WCOSS2 and C6 with the following or similar errors:

^[[38;21m2025-11-25 18:12:11,253 - INFO     - chem_fire_emission: Processing historical emissions for 2021-03-22 12:00:00 to 2021-03-30 00:00:00^[[0m
Traceback (most recent call last):
  File "/lfs/h2/emc/ptmp/david.huber/gw_3713/scripts/exglobal_prep_emissions.py", line 23, in <module>
    fireemis.initialize()
  File "/lfs/h2/emc/ptmp/david.huber/gw_3713/sorc/wxflow/src/wxflow/logger.py", line 252, in wrapper
    retval = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/lfs/h2/emc/ptmp/david.huber/gw_3713/ush/python/pygfs/task/chem_fire_emission.py", line 119, in initialize
    aero_emis_fire_version = str(self.task_config.AERO_EMIS_FIRE_VERSION)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lfs/h2/emc/ptmp/david.huber/gw_3713/sorc/wxflow/src/wxflow/attrdict.py", line 79, in __getattr__
    return self.__getitem__(item)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/lfs/h2/emc/ptmp/david.huber/gw_3713/sorc/wxflow/src/wxflow/attrdict.py", line 84, in __missing__
    raise KeyError(name)
KeyError: 'AERO_EMIS_FIRE_VERSION'

It looks like there is a key missing from a dictionary. Can you check this out and test it locally, please?

@DavidHuber-NOAA DavidHuber-NOAA added CI-Wcoss2-Failed CI testing on WCOSS for this PR has failed CI-Gaeac6-Failed **Bot use only** CI testing on Gaea C6 for this PR has failed and removed CI-Wcoss2-Running CI testing on WCOSS for this PR is in-progress CI-GaeaC6-Running (CM) CI testing is being run locally on Gaea C6. labels Nov 25, 2025
@bbakernoaa
Copy link
Contributor Author

@DavidHuber-NOAA yes I'll test it out asap

@bbakernoaa
Copy link
Contributor Author

@DavidHuber-NOAA There was two small errors that have been corrected. I did a run on gaeac6 using the ci

./generate_workflows.sh -A bil-fire3 -C -c /gpfs/f6/bil-fire3/world-shared/Barry.Baker/EXPTS/

All jobs passed except the arch since i don't have permission to the default archive directory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI-Gaeac6-Failed **Bot use only** CI testing on Gaea C6 for this PR has failed CI-Wcoss2-Failed CI testing on WCOSS for this PR has failed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GCAFSv1 Input data Add NEXUS as a submodule to the global-workflow GCAFS Prep emissions - fire update

7 participants