Issue with HAFS Workflow Stopping after atm_prep on NCAR Derecho Machine #282
-
|
Hi everyone, I am PhD student from University of Houston, and currently working on HAFS. I have successfully built and installed HAFS in NCAR derecho machine. But while running HAFS_rt_regional_atm, the workflow stops after running atm_prep. I have attached the screenshot of hafs_rt_status here, where it shows atm_prep is successful. but from hafs_atm_prep.log, I can see some following errors. I have also attached the log file here. I would greatly appreciate any insights or suggestions on how to resolve this issue. Thank you in advance for your help! Best regards, Screenshot: log_file: |
Beta Was this translation helpful? Give feedback.
Replies: 13 comments 1 reply
-
|
It looks like prod_util is not loaded. Could you check if you have prod_util installed on Derecho and loaded for your job? |
Beta Was this translation helpful? Give feedback.
-
|
Hi, thanks for your suggestion. I couldn't find prod_util installed on the NCAR Derecho system, but I found and installed NCEPLIBS-prod_utils from GitHub. Now, atm_prep runs without any error. However, the Rocoto workflow still does not start the next step, "ATM_IC." I couldn't find any log file to check its status. Does this step need any additional input files to run, and what are these? If not, can you suggest any reasons why the "ATM_IC" task in the Rocoto workflow might not start and where I should look to solve this? I am attaching the workflow XML file, hafs_atm_prep.log file, and cronjob script here. Best regards, Attached files:hafs_atm_prep.log |
Beta Was this translation helpful? Give feedback.
-
|
I don't have Derecho access. Run "rocotocheck" to see why the workflow is not moving. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, Thank you for pointing out the "rocotocheck" option. I am new to using the Rocoto workflow, and your guidance has been incredibly helpful. I've attached the output from the rocotocheck command below. It seems that the file /glade/derecho/scratch/mkhondaker/project/momen/murad/project3/noscrub/hafs-input/COMGFSv16/gfs.20200825/12/atmos/gfs.t12z.atmanl.nc does not exist. I attempted to download it, but I was only able to access the last 10 days of GFS files. Could you suggest where I might find these files for previous years (i.e., 2019-2023)? Thanks again for your assistance. Best regards, rocotocheck -v -w hafs-HAFS_rt_regional_atm-13L-2020082512.xml -d hafs-HAFS_rt_regional_atm-13L-2020082512.db -c all -t atm_ic Task: atm_ic Cycle: 202008251200 Job: This task has not been submitted for this cycle Task can not be submitted because: |
Beta Was this translation helpful? Give feedback.
-
|
The 2023 data is archived on Jet/Orion/Hercules platforms. HPSS will have 2023 and prior to 2023 data |
Beta Was this translation helpful? Give feedback.
-
|
Hi, |
Beta Was this translation helpful? Give feedback.
-
|
You can find GFS/GDAS/ENKF/RTOFS data to run HAFS here on Jet:
I am not aware of any documentation available on how to retrieve the needed data to run HAFS from HPSS. |
Beta Was this translation helpful? Give feedback.
-
|
Just chiming in on this discussion since there seems to be some confusion: the RDHPC HPSS referred to above is different from the NCAR HPSS. You can not access the hafs-input data directly from Derecho, you need an RDHPCS (NOAA HPC) account to access this data. There is a long archive of GFS input data on AWS in GRIB2 format (https://registry.opendata.aws/noaa-gfs-bdp-pds/), but from what I can tell HAFS expects netcdf input files. @BijuThomas-NOAA Is there any way to run HAFS with GRIB2 data as input? |
Beta Was this translation helpful? Give feedback.
-
|
To answer the questions above:
1. If DA is turned off (run_gsi=no), the AWS website includes all GFS files
required to run HAFS, including the GFS netcdf files, gfs.t??z.atmanl.nc
and gfs.t??z.sfcanl.nc for initial conditions;
2. If you change config. option from "ictype=gfsnetcdf" to
"ictype=gfsgrib2ab_0p25", you should be able to run HAFS with GRIB2 data as
input
…On Thu, Aug 8, 2024 at 2:52 PM Michael Kavulich ***@***.***> wrote:
Just chiming in on this discussion since there seems to be some confusion:
the RDHPC HPSS referred to above is different from the NCAR HPSS. You can
not access the hafs-input data directly from Derecho, you need an RDHPCS
<https://www.noaa.gov/organization/information-technology/getting-hpc-account>
(NOAA HPC) account to access this data.
There is a long archive of GFS input data on AWS in GRIB2 format (
https://registry.opendata.aws/noaa-gfs-bdp-pds/), but from what I can
tell HAFS expects netcdf input files. @BijuThomas-NOAA
<https://github.com/BijuThomas-NOAA> Is there any way to run HAFS with
GRIB2 data as input?
—
Reply to this email directly, view it on GitHub
<#282 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE73MQXGOIXOUGET6LZJJALZQO47RAVCNFSM6AAAAABLXFG4EGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMRXHA4TMNI>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
Thank you all for your help with the input file sources. I successfully tested the AWS GFS files and was able to use them without any issues. For the ocean model forcing, I found a similar source for RTOFS files here: RTOFS Files on AWS. However, I wasn't able to find RTOFS data prior to 2024. Could you please advise on how I can access the RTOFS data for the years 2021-2023? Unfortunately, I don't have access to NOAA machines. Thank you in advance! |
Beta Was this translation helpful? Give feedback.
-
|
Someone suggested that this site might have RTOFS data, but I'm not sure how much data is there or whether the site is still updated: https://polar.ncep.noaa.gov/global/data_access.shtml |
Beta Was this translation helpful? Give feedback.
-
|
Yes, RTOFS started being uploaded to AWS in 2024. I don't know if there are
any other ways to get RTOFS data prior to 2024 unless you have access to
NOAA computer systems.
…-Zhan
On Wed, Aug 28, 2024 at 12:19 PM Murad Khondaker ***@***.***> wrote:
Thank you all for your help with the input file sources. I successfully
tested the AWS GFS files and was able to use them without any issues. For
the ocean model forcing, I found a similar source for RTOFS files here: RTOFS
Files on AWS <https://noaa-nws-rtofs-pds.s3.amazonaws.com/index.html>.
However, I wasn't able to find RTOFS data prior to 2024. Could you please
advise on how I can access the RTOFS data for the years 2021-2023?
Unfortunately, I don't have access to NOAA machines.
Thank you in advance!
—
Reply to this email directly, view it on GitHub
<#282 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE73MQTVBUACXYXDGEO3ZPDZTX2BXAVCNFSM6AAAAABLXFG4EGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANBXG42TKMI>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Hello Gillian,
Using the link you pasted, I got an ftp link (
ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/rtofs/prod/) where you could
get some RTOFS data. Unfortunately I could not open it with my browser so I
can not tell you if it is updating or not. But in the past I used to get
the RTOFS output from there, but it only had about three days of the most
recent runs.
Best,
Maria
…----
Maria Aristizabal Vargas, She/Her
Physical Scientist lll, Lynker at NOAA/EMC
NOAA Center for Weather & Climate Prediction
5830 University Research Court
College Park, MD 20740
On Thu, Sep 19, 2024 at 1:04 PM Gillian Petro ***@***.***> wrote:
Someone suggested that this site might have RTOFS data, but I'm not sure
how much data is there or whether the site is still updated:
https://polar.ncep.noaa.gov/global/data_access.shtml
—
Reply to this email directly, view it on GitHub
<#282 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADBZRSKYZGVYLJGUOSILOQLZXL7YXAVCNFSM6AAAAABLXFG4EGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANRZGU4TAMY>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
Currently, HAFS application does not support Derecho. With that, I am closing this discussion thread for now. |
Beta Was this translation helpful? Give feedback.
It looks like prod_util is not loaded. Could you check if you have prod_util installed on Derecho and loaded for your job?