Clean up and remove prepoceanobs components #1887

AndrewEichmann-NOAA · 2025-09-09T20:17:32Z

Description

Removes guts of prepoceanobs for conversion to obsForge

Companion PRs

Should wait until data population NOAA-EMC/global-workflow#4031 is complete

Issues

Partially resolves #1876

Automated CI tests to run in Global Workflow

…with files from obsforge marine obs processing

…se files for real-time, GFSv17 prototypes are added with a gfsv17 suffix and is intended to run with obsforge processed marine obs. Finally, the switch for obsforge processing is set to False as the default.

Updating this feature branch to be up-to-date with develop

…ilable to use and changes the ioda netcdf suffix to .nc for the preparing the ocean obs

…o obsforge marine obs processing

guillaumevernieres

we can 💥 this too https://github.com/NOAA-EMC/GDASApp/tree/develop/utils/obsproc

parm/marine/obsprep/obsprep_config_gfsv17.yaml

JohnSteffen-NOAA

Looks good. Everything is consistent with the changes in feature/use_obsforge. All that's needed is to address the coding norm and unit test checks.

parm/marine/obs/obs_list_gfsv17.yaml.j2

apchoiCMD

Look seamless

emcbot · 2025-09-10T22:30:20Z

Automated GW-GDASApp Testing Results:
Machine: orion

Start: Wed Sep 10 04:18:45 PM CDT 2025 on orion-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Wed Sep 10 05:11:52 PM CDT 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E C96_gcafs_cycled|atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmsnowDA|C96_gcafs_cycled|C96C48_hybatmDA
Tests:                                  *Failed*
Tests: Failed at Wed Sep 10 05:30:20 PM CDT 2025
Tests: 97% tests passed, 1 tests failed out of 34
	2141 - test_gdasapp_C48mx500_hybAOWCDA_enkfgdas_marineanlletkf_202103250000 (Failed)
Tests: see output at /work2/noaa/da/role-da/CI/orion/GDASApp/workflow/PR/1887/global-workflow/sorc/gdas.cd/build/log.ctest

AndrewEichmann-NOAA · 2025-09-10T23:02:37Z

marineanlletkf is the only one that failed

guillaumevernieres · 2025-09-11T13:46:57Z

Copy paste from the letkf log:

Traceback (most recent call last):                                                                                                                                           
  File "/work2/noaa/da/role-da/CI/orion/GDASApp/workflow/PR/1887/global-workflow/scripts/exglobal_marine_analysis_letkf.py", line 22, in <module>                            
    MarineLetkf.initialize()                                                                                                                                                 
  File "/work2/noaa/da/role-da/CI/orion/GDASApp/workflow/PR/1887/global-workflow/sorc/wxflow/src/wxflow/logger.py", line 252, in wrapper                                     
    retval = func(*args, **kwargs)                                                                                                                                           
             ^^^^^^^^^^^^^^^^^^^^^                                                                                                                                           
  File "/work2/noaa/da/role-da/CI/orion/GDASApp/workflow/PR/1887/global-workflow/ush/python/pygfs/task/marine_letkf.py", line 128, in initialize                             
    jcb_config['observations'] = parse_j2yaml(obsconfigfile, jcb_config)                                                                                                     
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                     
  File "/work2/noaa/da/role-da/CI/orion/GDASApp/workflow/PR/1887/global-workflow/sorc/wxflow/src/wxflow/yaml_file.py", line 183, in parse_j2yaml                             
    raise FileNotFoundError(f"Input j2yaml file {path} does not exist!")                                                                                                     
FileNotFoundError: Input j2yaml file /work2/noaa/da/role-da/CI/orion/GDASApp/workflow/PR/1887/global-workflow/parm/gdas/marine/obs/obs_list_base.yaml.j2 does not exist!

…DASApp into feature/clean_prepobs

emcbot · 2025-09-11T19:28:26Z

Automated GW-GDASApp Testing Results:
Machine: orion

Start: Thu Sep 11 01:19:50 PM CDT 2025 on orion-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Thu Sep 11 02:14:29 PM CDT 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E C96_gcafs_cycled|atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmsnowDA|C96_gcafs_cycled|C96C48_hybatmDA
Tests:                                 *SUCCESS*
Tests: Completed at Thu Sep 11 02:28:26 PM CDT 2025
Tests: 100% tests passed, 0 tests failed out of 33

AndrewEichmann-NOAA · 2025-09-11T19:38:39Z

Note that this will cause CI to fail until NOAA-EMC/global-workflow#4031 is completed

guillaumevernieres

Thanks @AndrewEichmann-NOAA . Looks good.

guillaumevernieres and others added 11 commits July 25, 2025 13:33

use obsforge

02e0419

A couple of bugfixes to make obsforge work correctly

a001d66

Updates the obs list, obs config, and prep_ocean_obs scripts to work …

c238531

…with files from obsforge marine obs processing

Reverts the obs_list and obsprep yamls back to what's in develop. The…

2389023

…se files for real-time, GFSv17 prototypes are added with a gfsv17 suffix and is intended to run with obsforge processed marine obs. Finally, the switch for obsforge processing is set to False as the default.

Merge remote-tracking branch 'origin/develop' into feature/use_obsforge

bcdbeaf

Updating this feature branch to be up-to-date with develop

Fix coding norms

95fe8d0

Addresses reviewer comments on keeping the rads_adt_all obs space ava…

3d06101

…ilable to use and changes the ioda netcdf suffix to .nc for the preparing the ocean obs

Merge branch 'develop' into feature/use_obsforge

c7f7db8

Updates this branch to current develop and includes changes related t…

89642c4

…o obsforge marine obs processing

remove old method calls

b4b5ac1

remove bits of prepoceanobs

b544bd5

AndrewEichmann-NOAA requested review from guillaumevernieres, JohnSteffen-NOAA and apchoiCMD September 9, 2025 20:19

remove marine obs test material

4ee198e

guillaumevernieres reviewed Sep 9, 2025

View reviewed changes

parm/marine/obsprep/obsprep_config_gfsv17.yaml Outdated Show resolved Hide resolved

remove obs utils

34936cf

JohnSteffen-NOAA previously approved these changes Sep 10, 2025

View reviewed changes

parm/marine/obs/obs_list_gfsv17.yaml.j2 Outdated Show resolved Hide resolved

apchoiCMD previously approved these changes Sep 10, 2025

View reviewed changes

more removals

06badb4

AndrewEichmann-NOAA dismissed stale reviews from apchoiCMD and JohnSteffen-NOAA via 06badb4 September 10, 2025 17:14

AndrewEichmann-NOAA added 3 commits September 10, 2025 15:24

style and remove ioda converters

5a5420b

fixing linter

855de51

fixing linter

e62a671

AndrewEichmann-NOAA mentioned this pull request Sep 10, 2025

IGNORE: temporary PR for GDASApp CI NOAA-EMC/global-workflow#4047

Closed

22 tasks

Merge branch 'develop' into feature/clean_prepobs

f281329

AndrewEichmann-NOAA marked this pull request as ready for review September 10, 2025 21:01

AndrewEichmann-NOAA self-assigned this Sep 10, 2025

AndrewEichmann-NOAA added the orion-GW-RT Queue for automated testing with global-workflow on Orion label Sep 10, 2025

AndrewEichmann-NOAA added hera-GW-RT Queue for automated testing with global-workflow on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Sep 10, 2025

emcbot added orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed orion-GW-RT Queue for automated testing with global-workflow on Orion labels Sep 10, 2025

emcbot added orion-GW-RT-Failed Automated testing with global-workflow failed on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Sep 10, 2025

AndrewEichmann-NOAA requested review from apchoiCMD, JohnSteffen-NOAA and guillaumevernieres September 10, 2025 23:03

AndrewEichmann-NOAA added 2 commits September 11, 2025 13:09

fix, then get rid of letkf test

4c5b11a

Merge branch 'feature/clean_prepobs' of https://github.com/NOAA-EMC/G…

c67dd4b

…DASApp into feature/clean_prepobs

AndrewEichmann-NOAA added orion-GW-RT Queue for automated testing with global-workflow on Orion and removed orion-GW-RT-Failed Automated testing with global-workflow failed on Orion labels Sep 11, 2025

emcbot added orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed orion-GW-RT Queue for automated testing with global-workflow on Orion labels Sep 11, 2025

emcbot added orion-GW-RT-Passed Automated testing with global-workflow successful on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Sep 11, 2025

apchoiCMD approved these changes Sep 11, 2025

View reviewed changes

Merge branch 'develop' into feature/clean_prepobs

a1aca22

JohnSteffen-NOAA approved these changes Sep 11, 2025

View reviewed changes

guillaumevernieres approved these changes Sep 11, 2025

View reviewed changes

AndrewEichmann-NOAA mentioned this pull request Sep 12, 2025

Add dependency for prepoceanobs on obsForge status files NOAA-EMC/global-workflow#4054

Open

22 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Clean up and remove prepoceanobs components #1887

Clean up and remove prepoceanobs components #1887

Uh oh!

AndrewEichmann-NOAA commented Sep 9, 2025

Uh oh!

guillaumevernieres left a comment

Uh oh!

Uh oh!

JohnSteffen-NOAA left a comment •

edited

Loading

Uh oh!

Uh oh!

apchoiCMD left a comment

Uh oh!

emcbot commented Sep 10, 2025

Uh oh!

AndrewEichmann-NOAA commented Sep 10, 2025

Uh oh!

guillaumevernieres commented Sep 11, 2025

Uh oh!

emcbot commented Sep 11, 2025

Uh oh!

AndrewEichmann-NOAA commented Sep 11, 2025

Uh oh!

guillaumevernieres left a comment

Uh oh!

Uh oh!

Clean up and remove prepoceanobs components #1887

Are you sure you want to change the base?

Clean up and remove prepoceanobs components #1887

Uh oh!

Conversation

AndrewEichmann-NOAA commented Sep 9, 2025

Description

Companion PRs

Issues

Automated CI tests to run in Global Workflow

Uh oh!

guillaumevernieres left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JohnSteffen-NOAA left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

apchoiCMD left a comment

Choose a reason for hiding this comment

Uh oh!

emcbot commented Sep 10, 2025

Uh oh!

AndrewEichmann-NOAA commented Sep 10, 2025

Uh oh!

guillaumevernieres commented Sep 11, 2025

Uh oh!

emcbot commented Sep 11, 2025

Uh oh!

AndrewEichmann-NOAA commented Sep 11, 2025

Uh oh!

guillaumevernieres left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JohnSteffen-NOAA left a comment •

edited

Loading