Issue #547: Remove mask from read #598

mo-gill · 2025-07-17T14:56:46Z

Closes #547

…ate functions.

…sts.

cdds/cdds/common/plugins/cmip6_plus/data/model/HadGEM3-GC31-LL.json

matthew-mizielinski

A few tweaks to JSON files and docs

cdds/cdds/common/plugins/cordex/data/model/HadREM3-GA7-05.json

cdds/cdds/common/plugins/gcmodeldev/data/model/HadGEM3-GC31-LL.json

docs/tutorials/json_model_parameter.md

cdds/cdds/tests/test_validate/data/HadGEM3-GC31-HH.json

cdds/cdds/tests/test_validate/data/functional_model_params/invalid/UKESM1-0-LL.json

cdds/cdds/tests/test_validate/data/functional_model_params/valid/UKESM1-0-LL.json

cdds/cdds/tests/test_common/test_request/data/test_request_cordex.cfg

matthew-mizielinski

A few small things

cdds/cdds/configure/user_config.py

cdds/cdds/tests/test_plugins/test_cmip6/data/model/HadGEM3-GC31-LL.json

cdds/cdds/tests/test_plugins/test_cmip6_plus/data/model/HadGEM3-GC31-LL.json

cdds/cdds/tests/test_plugins/test_cordex/data/model/HadREM3-GA7-05.json

cdds/cdds/tests/test_plugins/test_gcmodeldev/data/model/HadGEM3-GC31-LL.json

cdds/cdds/tests/test_validate/data/HadGEM3-GC31-HH.json

cdds/cdds/tests/test_validate/data/HadGEM3-GC31-HH_invalid.json

cdds/cdds/tests/test_validate/data/HadGEM3-GC31-HH_warning.json

… into 547_remove_mask_from_read

cdds/cdds/configure/user_config.py

matthew-mizielinski

LGTM and after 9b92377 the tests pass.

As discussed, please now retarget this change at main. possibly through a separate PR

This change moves the halo removal from the extract filtering step to mip_convert.

#547: Attempt at using masking for subsetting/masking ocean data.

c6bbdc1

mo-gill added this to the CDDS v3.2.1 milestone Jul 17, 2025

mo-gill self-assigned this Jul 17, 2025

mo-gill added 11 commits July 17, 2025 14:59

#547: Remove a comment.

8cc9807

#547: Add comments.

b1a5f14

#547: Fix some formatting.

dcbb90f

#547: Add halo removal for 2D ocean data. Tidy up code.

4988f98

#547: Starting to try to separate subsetting of dimensions into separ…

3b83370

…ate functions.

#547: Further attempt at subsetting different dimensions.

b60c6e8

#547: Add halo_remove detection - non-functional with zostoga data.

14e1e50

#547: Add another log message, think removal working with zostoga now.

bd116a9

#547: Remove comment and some whitespace.

a07fa41

#547: Minor adjustments to _apply removal.

7c9bbd7

#547: First attempt at adding defaults. Non-functional.

8e305c3

mo-jareddrayton modified the milestones: CDDS v3.2.1, CDDS v3.2.2 Aug 27, 2025

mo-gill added 14 commits August 29, 2025 09:22

#547: Further attempt to implement default values.

ab5db16

#547: Trying to adjust logic for halo removal attributes.

0a84ac4

#547: Remove unnecessary data from jsons.

f2e4a00

#547: Add halo removal overloading.

e034822

#547: Further attempt to implement default values for plugins.

9a72f09

#547: Default retrieval semi-working.

d85c26a

#547: Default working, logic needs work.

0b2ff04

#547: Add default values to other mip config jsons.

cb98f82

#547: Amend altered indentation, remove a breakpoint.

29f6a09

#547: Remove unnecessary code, run pep8.

4e3b858

#547: Minor reformatting, replace hardcoded string logic.

f318a7a

#547: Add abstract property to class fix mypy err.

07e0e65

#547: Fix a pep8 fail.

1bced7a

#547: Add longitude check for diaptr.

1aacdb0

mo-gill added 8 commits September 16, 2025 08:29

#547: Remove default_halo_removal from jsons.

e8a312b

#547: Remove code related to halo_options.

2358d99

#547: Remove some halo sections from the docs.

762d16e

#547: Remove code linked to halo_removal_latitude/longitude

1b72ccd

#547: Revert deletion of misc section validation code.

b20d60f

#547: Remove halo removal validation from MiscSectionValidator and te…

70b3769

…sts.

#547: Amend docs in json_model_parameter page.

2804a50

#547: Amend halo_removal section in mip_convert docs.

029297a

mo-gill requested a review from matthew-mizielinski September 17, 2025 08:20

matthew-mizielinski reviewed Sep 17, 2025

View reviewed changes

cdds/cdds/common/plugins/cmip6_plus/data/model/HadGEM3-GC31-LL.json Show resolved Hide resolved

#547: Add halo_removal section in cmip6 plus model parameters JSON

ec45485

matthew-mizielinski reviewed Sep 17, 2025

View reviewed changes

mo-gill added 7 commits September 18, 2025 07:35

#547: Add empty halo removal section.

1d81a83

#547:correction to location of halo removal in cmip6 json.

24292c8

#547: Add halo_removal section to gcmodeldev jsons.

de38be1

#547: Amend json model docs.

9f68a3d

#547: Amend halo_removal docs formatting.

744affc

#547: Add halo removal section to test jsons.

92c4485

#547: Add cordex halo_removal values for apm,ap6,ap5,ap7 in json

3f94f8f

mo-gill requested a review from matthew-mizielinski September 19, 2025 08:58

matthew-mizielinski reviewed Sep 19, 2025

View reviewed changes

mo-gill added 3 commits September 19, 2025 10:10

#547: Remove argument and a logger.

b111f5f

#547: Add more halo_removal sections to jsons.

e9c46ef

Merge branch '547_remove_mask_from_read' of github.com:MetOffice/CDDS…

e6818ed

… into 547_remove_mask_from_read

mo-gill requested a review from matthew-mizielinski September 19, 2025 11:03

matthew-mizielinski reviewed Sep 19, 2025

View reviewed changes

cdds/cdds/configure/user_config.py Outdated Show resolved Hide resolved

#547 pep8 fix

9b92377

matthew-mizielinski approved these changes Sep 19, 2025

View reviewed changes

mo-gill mentioned this pull request Sep 22, 2025

Merge halo removal work via rebase #633

Merged

mo-gill added a commit that referenced this pull request Sep 30, 2025

#547: Merge halo removal work via rebase (#633) (#598)

325193b

This change moves the halo removal from the extract filtering step to mip_convert.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Issue #547: Remove mask from read #598

Issue #547: Remove mask from read #598

Uh oh!

mo-gill commented Jul 17, 2025

Uh oh!

Uh oh!

matthew-mizielinski left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matthew-mizielinski left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matthew-mizielinski left a comment

Uh oh!

Uh oh!

Issue #547: Remove mask from read #598

Are you sure you want to change the base?

Issue #547: Remove mask from read #598

Uh oh!

Conversation

mo-gill commented Jul 17, 2025

Uh oh!

Uh oh!

matthew-mizielinski left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matthew-mizielinski left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matthew-mizielinski left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!