-
Notifications
You must be signed in to change notification settings - Fork 75
Add example of regional configuration using regional-mom6 package forced with ACCESS-OM2-01 #338
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
We need to ensure the example runs. Also we need to modify so that it uses the cosima-cookbook to load ACCESS-OM2 data; see COSIMA/regional-mom6#131 |
@ashjbarnes I tried to run this and it errors on initial condition. See the output of |
@ashjbarnes is this error related to the bug in COSIMA/regional-mom6#170? If so, it means that https://github.com/COSIMA/regional-mom6/releases/tag/v0.5.1 would be required. cc @luweiyang |
I tried to see if regional-mom6 v0.5.2 healed the issues I was having. It didn't... :( |
@ashjbarnes I tried to run the ACCESS-OM2 forced case and realised that the west and east unprocessed boundary files |
I've fixed the west and east boundary condition issues. I changed cell 6 in step 2 (changes in bold): The notebook is running fine now and I've pushed the working version to the However, I couldn't run the experiment as I keep getting warning & error messages about initialization and extreme surface state. So somehow the date of ocean forcing and atmospheric forcing do not match? Since we use repeated year forcing, I didn't think that would be a problem... Any thoughts? And it turns out that the temperature unit is K (instead of C) in
|
Great! something else we’d want to do is to use the cookbook to load the ACCESS-OM2 output |
@@ -0,0 +1,699 @@ | |||
{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why there are 3 commented out lines of code @luweiyang?
if we need them, then uncomment,
if we don't need them then delete?
Reply via ReviewNB
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we need them. L20 and L21 are the same as L6 and L7 for selecting data along the dims yu_ocean
and yt_ocean
. I will delete them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally, commented code is bad practice...
Either we want the code or we delete it.
It's good to let go. :)
@@ -0,0 +1,699 @@ | |||
{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, I commented them out when I was debugging because at some stage I did get some client-related errors. But it's no longer an issue. I will uncomment them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, do we need to add this back or not?
this is related to loading from cookbook: COSIMA/regional-mom6#131 |
Yes, I checked the date as well and was confused. As @angus-g pointed out, the data in this folder The info for the atmospheric forcing is provided in Does this mismatch in year explain the initialisation error? |
Yes good find! It'll be crashing because of the temperatures being in Kelvin. This is strange - these lines of code check the minimum surface temperature value of the dataset and then assume that it's in Kelvin if this minimum is above 100. This used to work but apparently not any more! The code here hasn't been changed for a long time This notebook worked fine a while ago but it was neglected whilst we made a lot of changes. Somewhere we've introduced a change that breaks it but I don't know where.. |
This is good. It will force us to find whe K->C doesn’t work and implement a test to capture future similar failures. Let’s not ruminate on the past. Sure it worked at some point. It doesn’t matter that much, unless it helps us find why it’s not working now. Let’s just make an example that works now. It doesn’t even have to resemble how it was in the past if we want to showcase something else. |
If we are gonna load the RYF then there is no point referring to “year 2003” or anything. All years are the same! They are looped. I do think it more intuitive if we force with IAF, some particular year that we load via the cookbook |
Knowing that it worked in the past should help us troubleshoot though right? Nothing about the forcing dataset has changed, only our code. If our code used to correctly convert K to C and no longer does, that means something in the package (specifically something in the initial condition method) has changed that prevents it from doing this. I've had a quick look in case there was a premature "fill nans with 0s" or something but that doesn't seem to be the case |
Yes, same ocean forcing year and same atmos forcing did work in the past. I remember a while ago @ashjbarnes updated and recompiled MOM6, and with new exe file everything worked fine. |
I never doubted that it worked in the past.. But now it's not, so let's fix it. There are seem to be at least 1 or more issues. One is that temperature is not converted to K->C despite there is a method that supposedly does that. Possibly:
How does ERA5-forced example work? Is ERA5 in degrees C to begin with? |
This statement is unclear to me. I don't get what you want to say here @ashjbarnes. |
Well done adding the buffer to the initial conditions, @navidcy ! Where did you make changes (which lines)? It was running! After 50 mins, it crashed because of the following error:
In the notebook, I've include comments about the version of regional-mom6 (v0.6.0) that users should load and added cells to plot initial conditions. Once we fix the bug about the time, it will be ready to be included in the demos. |
We'd also like to replace om2_input = xr.open_mfdataset(
f"/g/data/ik11/outputs/access-om2-01/01deg_jra55v13_ryf9091/output1077/ocean/ocean_daily*.nc",
parallel=True, compat='override', chunks='auto')[["u", "v", "salt", "temp", "eta_t"]].sel(
yu_ocean = slice(latitude_extent[0] - buffer, latitude_extent[1] + buffer),
yt_ocean = slice(latitude_extent[0] - buffer, latitude_extent[1] + buffer)).isel(time = slice(0, 5)) with using the cookbook to load this data. Could we do that? |
You mean intake, right? |
Yes, let's do intake! |
@anton-seaice or @dougiesquire, could you help us here? we'd like to replace: om2_input = xr.open_mfdataset(
f"/g/data/ik11/outputs/access-om2-01/01deg_jra55v13_ryf9091/output1077/ocean/ocean_daily*.nc",
parallel=True, compat='override', chunks='auto')[["u", "v", "salt", "temp", "eta_t"]].sel(
yu_ocean = slice(latitude_extent[0] - buffer, latitude_extent[1] + buffer),
yt_ocean = slice(latitude_extent[0] - buffer, latitude_extent[1] + buffer)).isel(time = slice(0, 5)) with intake. I tried import intake
catalog = intake.cat.access_nri
ds = catalog["01deg_jra55v13_ryf9091"].search(
variable=["u", "v", "salt", "temp", "eta_t"],
frequency='1day').to_dask() but got ---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[13], line 6
1 import intake
2 catalog = intake.cat.access_nri
4 ds = catalog["01deg_jra55v13_ryf9091"].search(
5 variable=["u", "v", "salt", "temp", "eta_t"],
----> 6 frequency='1day').to_dask()
File [/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.07/lib/python3.10/site-packages/intake_esm/core.py:811](http://localhost:8890/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.07/lib/python3.10/site-packages/intake_esm/core.py#line=810), in esm_datastore.to_dask(self, **kwargs)
796 """
797 Convert result to an xarray dataset.
798
(...)
808 :py:class:`~xarray.Dataset`
809 """
810 if len(self) != 1: # quick check to fail more quickly if there are many results
--> 811 raise ValueError(
812 f'Expected exactly one dataset. Received {len(self)} datasets. Please refine your search or use `.to_dataset_dict()`.'
813 )
814 res = self.to_dataset_dict(**{**kwargs, 'progressbar': False})
815 if len(res) != 1: # extra check in case kwargs did modify something
ValueError: Expected exactly one dataset. Received 9 datasets. Please refine your search or use `.to_dataset_dict()`. |
Because each variable is stored in a different file, intake will return multiple datasets. So you want: import intake
catalog = intake.cat.access_nri
esmds = catalog["01deg_jra55v13_ryf9091"]
ds_dict = esmds.search(
path=".*output1077.*",
variable=["u", "v", "salt", "temp", "eta_t"],
frequency='1day',
).to_dataset_dict()
ds = xr.merge(ds_dict.values(), join="inner") |
can I even drop the |
I tried the above and after 10min nothing was happening... I think the dask cluster dropped. :( |
These particular data are actually quite problematic.
If you're wanting a command that opens a block of consistent, global data, you could filter out the ds_dict = esmds.search(
variable=["u", "v", "salt", "temp", "eta_t"],
frequency='1day',
filename=".*[a-z].nc"
).to_dataset_dict()
ds = xr.merge(ds_dict.values(), join="inner") Note, |
If you're only selecting 5 time points then it's definitely better not to open >10 years of daily data, which is what my previous suggested command does. If you don't like the ds_dict = esmds.search(
variable=["u", "v", "salt", "temp", "eta_t"],
frequency='1day',
start_date="2170-04-01, 00:00:00",
).to_dataset_dict()
ds = xr.merge(ds_dict.values(), join="inner") |
OK, I see your point. We just want few days of daily data for demo purposes. Nothing beyond that. @dougiesquire thanks for elaborating! Do you have, perhaps, a different dataset in mind that does not have these issues? RYF 0.1 degree? |
What's wrong with specifying the |
Nothing. That sounds great! I'll do that ;) |
Can I also use |
Done!!!! |
This is ready. Could somebody (have a look) and approve? |
Not in a useful way. These e.g.
returns the data assets from one file which starts on April 1
returns nothing because there is no file containing daily data which starts on the first of april AND finishes on the 9th of april. It doesn't search for all data in that range. If you understand regex you select a range by using a regex of start dates (something like this, untested:)
|
thanks @anton-seaice ! |
…es into ncc/regional-mom6-v2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't looked in detail at the whole notebook, but the intake stuff looks good.
I’ve scrutinised it many times and it runs. I’m merging and we can open a new PR to tweak if needed. |
This closes COSIMA/regional-mom6#59
cc @ashjbarnes