Skip to content

Conversation

rbeucher
Copy link
Contributor

@rbeucher rbeucher commented Jun 6, 2024

Following the discussion in issue #313, we propose converting the recipes to use Intake, given that the Cookbook is no longer supported and the ACCESS-NRI Intake catalog is now available.

A few months ago, @max-anu began working on this transition. This pull request contains the changes @max-anu made to the notebook specified in the title.

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:42Z
----------------------------------------------------------------

We should add the conda environment that we need to use to run it.


Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:43Z
----------------------------------------------------------------

I prefer to have a chunk that only loads libraries and any use of functions (line 5) I like to have in a separate chunk.

While trying to run the notebook in Gadi, I got an error in line 5 because I was not a member of project xp65. We should probably add this information to the introduction to make sure users can run the notebook.


navidcy commented on 2024-07-29T05:14:16Z
----------------------------------------------------------------

Agree

Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:43Z
----------------------------------------------------------------

When this address "tcp://10.6.77.34:8786" is used in the Client() , then the chunk does not run. Is this needed?


navidcy commented on 2024-07-29T05:14:27Z
----------------------------------------------------------------

I doubt

Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:44Z
----------------------------------------------------------------

This description does not appear to match what the chunk below is doing. Are you printing the variables available in the catalog?


navidcy commented on 2024-07-29T05:14:34Z
----------------------------------------------------------------

I rephrased

Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:45Z
----------------------------------------------------------------

This chunk does not run because cat_subset is not defined until the next chunk. I would replace with:

experiment = '025deg_jra55_iaf_omip2_cycle6'
cat_subset = catalog[experiment]
sorted(set().union(*cat_subset.df['variable']))

Then the chunk below will look like the following:

variable = 'sst'
var_search = cat_subset.search(variable=variable, frequency='1mon')
darray = var_search.to_dask()
darray = darray[variable]
SST = darray

navidcy commented on 2024-07-29T05:14:59Z
----------------------------------------------------------------

I doubt the sorted(set().union(*cat_subset.df['variable'])) is needed...

Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:45Z
----------------------------------------------------------------

We could probably replace the lines below:

darray = darray[variable]
SST = darray

with the following:

SST = darray[variable]

navidcy commented on 2024-07-29T05:15:09Z
----------------------------------------------------------------

of course!

Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:46Z
----------------------------------------------------------------

Since the units have not been updated, I would also add a note about the units in the colourbar are not correct.


Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:47Z
----------------------------------------------------------------

I would update the units after the conversion, so that is before creating the plot.

Also, I do not think this chunk is necessary because the units were already defined in the original dataset.


Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:47Z
----------------------------------------------------------------

I would update the name of the function from plot_global_temp_in_degrees_celcius to map_mean_temp_in_degrees_celcius because it works with a subset of the data array, not just globally (i.e., I could plot the Southern Ocean only although with a suboptimal projection) and it calculates the mean spatially to create a map (as opposed to mean over time for a time series).


navidcy commented on 2024-07-29T05:18:43Z
----------------------------------------------------------------

I disagree... the function will plot a global map if you provide with a global field, no?

Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:48Z
----------------------------------------------------------------

This experiment (OM4_025.JRA_RYF) is not available in the catalog, refer to ACCESS-NRI/access-nri-intake-catalog#175.

From this point onwards, the notebook does not run because either the experiments or variables used are not available in the catalogue.


navidcy commented on 2024-07-29T05:18:59Z
----------------------------------------------------------------

Sure...

Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:49Z
----------------------------------------------------------------

This experiment (025deg_jra55v13_iaf_gmredi6) is not available either. The 0.25 degree options available are:

025deg_jra55_iaf_omip2_cycle1
025deg_era5_iaf
025deg_jra55_iaf_omip2_cycle3
025deg_jra55_ryf9091_gadi
025deg_jra55_iaf_omip2_cycle2
025deg_jra55_iaf_omip2_cycle6
025deg_jra55_ryf_era5comparison
025deg_era5_ryf
025deg_jra55_iaf_era5comparison
025deg_jra55_iaf_omip2_cycle5
025deg_jra55_iaf_omip2_cycle4

Note that Tair_m is not available in any of the options above. Needs to be looked into.


Copy link

review-notebook-app bot commented Jul 1, 2024

View / edit / reply to this conversation on ReviewNB

lidefi87 commented on 2024-07-01T05:30:49Z
----------------------------------------------------------------

Line #4.    ice_air_temp = ice_air_temp.assign_coords({'TLON': ice_grid.tlon.pint.to('degrees_E'), 'TLAT': ice_grid.tlat.pint.to('degrees_N')})

This does not work because it is calling ice_air_temp which is not created because the Tair_m variable does not exist in any of the 0.25 degree datasets.


@lidefi87 lidefi87 self-requested a review July 1, 2024 22:07
@anton-seaice anton-seaice marked this pull request as draft July 2, 2024 06:17
Copy link
Collaborator

navidcy commented Jul 29, 2024

Agree


View entire conversation on ReviewNB

Copy link
Collaborator

navidcy commented Jul 29, 2024

I doubt


View entire conversation on ReviewNB

Copy link
Collaborator

navidcy commented Jul 29, 2024

I rephrased


View entire conversation on ReviewNB

Copy link
Collaborator

navidcy commented Jul 29, 2024

I doubt the sorted(set().union(*cat_subset.df['variable'])) is needed...


View entire conversation on ReviewNB

Copy link
Collaborator

navidcy commented Jul 29, 2024

of course!


View entire conversation on ReviewNB

Copy link
Collaborator

navidcy commented Jul 29, 2024

I disagree... the function will plot a global map if you provide with a global field, no?


View entire conversation on ReviewNB

Copy link
Collaborator

navidcy commented Jul 29, 2024

Sure...


View entire conversation on ReviewNB

@navidcy
Copy link
Collaborator

navidcy commented Jul 29, 2024

Thanks @lidefi87 for your comments. I took them into consideration and pushed... but unfortunately we can't continue with this PR until MOM6 output is part of intake...

@lidefi87
Copy link
Collaborator

lidefi87 commented Aug 6, 2024

No worries. I am happy that some of the comments were useful at least

@navidcy
Copy link
Collaborator

navidcy commented Aug 31, 2024

@dougiesquire, @anton-seaice do we have MOM6 output in the intake catalog or not yet?

@anton-seaice
Copy link
Collaborator

@dougiesquire, @anton-seaice do we have MOM6 output in the intake catalog or not yet?

Not yet. Sounds like its close - should be this month I would guess.

@marc-white
Copy link
Contributor

Hi team, there's enough data is in the Catalog now to finish off this notebook conversion, so I've made a first pass at doing so. Please re-check. (In particular, I had to select some data myself for the last section of the notebook on when things go wrong - please confirm if this is a suitable dataset or not.)

@rbeucher rbeucher marked this pull request as ready for review May 23, 2025 01:11
Copy link

review-notebook-app bot commented May 23, 2025

View / edit / reply to this conversation on ReviewNB

anton-seaice commented on 2025-05-23T03:03:43Z
----------------------------------------------------------------

Line #4.    var_search = cat_subset.search(variable=variable, frequency='1mon')

It would be more agnostics to search by standard name (sea_surface_temperature)


Copy link

review-notebook-app bot commented May 23, 2025

View / edit / reply to this conversation on ReviewNB

anton-seaice commented on 2025-05-23T03:03:44Z
----------------------------------------------------------------

What happened to the landmask? Showing the temperatures over land as -10C is confusing


@rbeucher rbeucher force-pushed the INTAKE_Model_Agnostic_Analysis branch from 82cab8d to dd00a5d Compare June 27, 2025 01:31
@rbeucher
Copy link
Contributor Author

I had a run through and it looks good to me. I have just dequantify the arrays before plotting as it was causing some pickle serialisation issues. @marc-white.

@marc-white
Copy link
Contributor

View / edit / reply to this conversation on ReviewNB

anton-seaice commented on 2025-05-23T03:03:44Z ----------------------------------------------------------------

What happened to the landmask? Showing the temperatures over land as -10C is confusing

@rbeucher do you still see this issue when you run the notebook? I'm seeing it, and I'm not sure why it's occurring. As far as I can tell, there's no explicit land-mask occurring in the code (i.e., it must be in the data), but I don't know what would have changed between the CC data and what was loaded into the Intake catalog.

@rbeucher
Copy link
Contributor Author

oh yes. Let me check that

@rbeucher
Copy link
Contributor Author

This is strange, I don't think itt has anything to do with the data. I would merge and open an issue to fix this later.

@marc-white marc-white self-requested a review June 27, 2025 04:40
@marc-white marc-white merged commit 09e843b into main Jun 27, 2025
3 checks passed
@marc-white marc-white deleted the INTAKE_Model_Agnostic_Analysis branch June 27, 2025 04:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

7 participants