Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
170 commits
Select commit Hold shift + click to select a range
0e2b1c3
begun refactor of internal utils
freemansw1 Sep 11, 2023
f12bd72
add new xarray utils file for xarray functions
freemansw1 Sep 11, 2023
e8b8c71
continuing to separate out utilities into iris and xarray
freemansw1 Sep 12, 2023
a480730
ensure py3.8 compatibility, ensure output types work
freemansw1 Sep 13, 2023
d6cd8cc
Merge branch 'internal_utils_reorg' into sfreeman_xarray_work
freemansw1 Sep 19, 2023
3c7404e
merging in new changes
freemansw1 Sep 19, 2023
612082b
Merge branch 'internal_utils_reorg' into sfreeman_xarray_work
freemansw1 Sep 19, 2023
94d6da3
Updates to xarray support in utilities.
freemansw1 Sep 20, 2023
a3b5751
Added additional tests to new xarray utilities
freemansw1 Sep 21, 2023
76690d3
switch feature_detection_multithreshold_timestep to xarray
freemansw1 Sep 21, 2023
110a205
first step of switch feature_detection_multithreshold to xarray
freemansw1 Sep 21, 2023
61329e4
linting for xarray_utils
freemansw1 Sep 21, 2023
c566788
adding new test function to test iris and xarray pathways
freemansw1 Sep 29, 2023
ae6cf80
working xarray interpolation now
freemansw1 Oct 4, 2023
3dbf497
adding 2D array interpolation
freemansw1 Oct 5, 2023
755b4ea
switch to asserting for almost equal
freemansw1 Oct 5, 2023
d5c354b
Merge branch 'segmentation_type_hints' into sfreeman_xarray_work
freemansw1 Oct 5, 2023
12b9bae
progress on segmentation xarray support
freemansw1 Oct 5, 2023
1ae63bb
Merge remote-tracking branch 'julia/feature_statistics' into sfreeman…
freemansw1 Oct 5, 2023
3170496
incorporating julia's statistics code
freemansw1 Oct 5, 2023
8dfc1dd
adding times to add_coordinates
freemansw1 Oct 7, 2023
d1d48ef
Merge branch 'sfreeman_xarray_work_fd' into sfreeman_xarray_work_fd_j…
freemansw1 Oct 11, 2023
f642823
Merge remote-tracking branch 'julia/feature_statistics' into sfreeman…
freemansw1 Oct 11, 2023
cb191b1
Merge remote-tracking branch 'upstream/RC_v1.5.x' into sfreeman_xarra…
freemansw1 Nov 8, 2023
413c999
Merge remote-tracking branch 'upstream/RC_v1.5.x' into sfreeman_xarra…
freemansw1 Nov 9, 2023
034982b
fixes for updates from upstream
freemansw1 Nov 9, 2023
0a5f9b7
fix literal import in feature detection
freemansw1 Nov 9, 2023
6c636ce
Merge remote-tracking branch 'upstream/RC_v1.5.x' into sfreeman_xarra…
freemansw1 Nov 13, 2023
561ad32
add new tests
freemansw1 Nov 15, 2023
87c5603
add RNG for vertical axis
freemansw1 Nov 21, 2023
6d12038
Updated coordinate interpolation for xarray, things now match.
freemansw1 Dec 2, 2023
aa71de8
Merge remote-tracking branch 'upstream/RC_v1.5.x' into sfreeman_xarra…
freemansw1 Dec 2, 2023
e763274
Merge remote-tracking branch 'upstream/RC_v1.5.x' into sfreeman_xarra…
freemansw1 Dec 4, 2023
752d285
Added preservation of iris datatypes based on whether or not iris dat…
freemansw1 Dec 4, 2023
f32dc69
Merge branch 'sfreeman_xarray_work' into xarray_segmentation
freemansw1 Dec 4, 2023
ddf7501
updates for xarray-based segmentation
freemansw1 Dec 4, 2023
3272512
move segmentation into its own folder.
freemansw1 Dec 4, 2023
4b6e1fd
add in folder to packages of setup.py
freemansw1 Dec 4, 2023
d66c27b
switched to swap_dims and skipped dataset conversion
freemansw1 Dec 5, 2023
14a249a
added specific functions to segmentation __init__.py
freemansw1 Dec 20, 2023
ca5631e
fix bug with numpy typing
freemansw1 Feb 7, 2024
8679d44
updates from @w-k-jones review
freemansw1 Feb 23, 2024
4d54d24
Refactors to address @w-k-jones comments
freemansw1 Feb 28, 2024
3c43b5e
Some things solved, but not all tests passing due to a headache aroun…
freemansw1 Feb 28, 2024
72bd78a
Merge remote-tracking branch 'upstream/RC_v1.5.x' into sfreeman_xarra…
freemansw1 Mar 12, 2024
d6469b1
resolving merge issues
freemansw1 Mar 12, 2024
23b103e
resolve issues with time
freemansw1 Mar 13, 2024
d182a44
Merge remote-tracking branch 'upstream/RC_v1.5.x' into xarray_segment…
freemansw1 Mar 13, 2024
b9da40a
fix merge issues
freemansw1 Mar 13, 2024
0d85a21
update codecov to use 3.11 to resolve xarray/iris issues
freemansw1 Mar 13, 2024
f279584
Merge remote-tracking branch 'upstream/RC_v1.6.0' into sfreeman_xarra…
freemansw1 Mar 27, 2024
0f4dcee
fix for renamed functions
freemansw1 Mar 27, 2024
6c16917
Merge branch 'sfreeman_xarray_work_fd_julia_avgs' into xarray_segment…
freemansw1 Apr 3, 2024
c54d5d5
fixed from Julia's comments
freemansw1 May 9, 2024
2ed9ab9
Merge branch 'sfreeman_xarray_work_fd_julia_avgs' into xarray_segment…
freemansw1 Jun 3, 2024
c07b1d5
fix preserve_iris_datetime_types for most cases.
freemansw1 Jul 17, 2024
cd19a20
updated add_coordinates_to_features to work with coordinates that var…
freemansw1 Jul 18, 2024
e6deecb
Merge branch 'sfreeman_xarray_work_fd_julia_avgs' into xarray_segment…
freemansw1 Jul 18, 2024
7f6930b
add keyword for preserve_iris_datetime_types to feature detection
freemansw1 Jul 23, 2024
5f82097
Merge pull request #439 from tobac-project/RC_v1.5.x
freemansw1 Jul 24, 2024
bbb6b67
Merge remote-tracking branch 'upstream/RC_v1.6.0' into sfreeman_xarra…
freemansw1 Jul 24, 2024
0ebee82
Merge pull request #354 from freemansw1/sfreeman_xarray_work_fd_julia…
freemansw1 Jul 25, 2024
084c42d
Merge remote-tracking branch 'upstream/RC_v1.6.0' into xarray_segment…
freemansw1 Jul 25, 2024
a4744e3
fix for odd ns/us datetime conversions. converting to datetime64 alwa…
freemansw1 Aug 30, 2024
2551094
Merge remote-tracking branch 'origin/xarray_segmentation_seg_refactor…
freemansw1 Sep 12, 2024
c5e9e34
updates to fix test issues
freemansw1 Oct 28, 2024
e39d736
fixes for older versions of xarray
freemansw1 Oct 28, 2024
beaf022
added time padding for segmentation
freemansw1 Oct 28, 2024
9a559ca
Merge remote-tracking branch 'upstream/main' into v160_merge_main
freemansw1 Oct 28, 2024
0be6ddf
fix merge conflicts and failed tests with v1.5.4
freemansw1 Oct 31, 2024
5a9190e
fix formatting
freemansw1 Oct 31, 2024
2a3cf62
fixes for addressing comments
freemansw1 Nov 13, 2024
6725463
got rid of attributes
freemansw1 Nov 13, 2024
82af748
fixed typing error
freemansw1 Nov 14, 2024
ad8d41a
Merge remote-tracking branch 'upstream/main' into v160_merge_main
freemansw1 Jan 6, 2025
848475d
Merge remote-tracking branch 'upstream/main' into v160_merge_main
freemansw1 Jan 6, 2025
62b6f26
add annotations
freemansw1 Jan 6, 2025
9d4f03e
add another future import
freemansw1 Jan 6, 2025
76fdd71
Merge pull request #463 from freemansw1/v160_merge_main
freemansw1 Jan 6, 2025
5243c90
Merge remote-tracking branch 'upstream/RC_v1.6.0' into xarray_segment…
freemansw1 Jan 9, 2025
2b8c03e
update decorators
freemansw1 Jan 9, 2025
1364012
black formatting
freemansw1 Jan 9, 2025
d6ee0a1
updates to 1xMxN vs MxN issue
freemansw1 Jan 24, 2025
e927458
fix for readthedocs deprecation
freemansw1 Mar 14, 2025
5238872
fix for notebook
freemansw1 Mar 14, 2025
6aa1130
black reformatting for new version
freemansw1 Mar 14, 2025
f05da21
Merge pull request #417 from freemansw1/xarray_segmentation_seg_refactor
freemansw1 Mar 15, 2025
63822df
Update model OLR notebook to load/save using xarray and add more visu…
w-k-jones Mar 16, 2025
4ea35c9
Fix test for coords in feature detection output that previously had n…
w-k-jones Mar 26, 2025
f893614
Fix bug with datetime conversions during feature detection and add test
w-k-jones Mar 26, 2025
e2bbfe6
Add test for datetime type in 3D data and with 360 day calendar
w-k-jones Mar 26, 2025
565c386
Formatting
w-k-jones Mar 26, 2025
1e43dfc
Fix terminology
w-k-jones Mar 26, 2025
6bcfd96
Update model OLR notebook to load/save using xarray and add more visu…
w-k-jones Mar 16, 2025
88e389c
Update animation_mask_field to use xarray
w-k-jones Mar 26, 2025
7608d61
Update OLR example with fixed animation
w-k-jones Mar 26, 2025
d9ee3cd
Fix merge conflict
w-k-jones Mar 26, 2025
e477741
Formatting
w-k-jones Mar 26, 2025
c507f21
Add a generators module and add generator for iterating over time thr…
w-k-jones Mar 27, 2025
7dd6543
Add check for time_var_name in field_and_features_over_time and test
w-k-jones Mar 27, 2025
2183b33
Update segmentation to use new field_and_features_over_time generator
w-k-jones Mar 27, 2025
d732def
Add generators module
w-k-jones Mar 27, 2025
db6d8d0
Add datetime conversion utils and tests
w-k-jones Mar 27, 2025
05bbdbd
Update bulk statistics to use field_and_features_over_time generator
w-k-jones Mar 27, 2025
1a65e2e
Update decorators, feature detection and coordinate interpolation so …
w-k-jones Mar 27, 2025
5651c7e
Formatting
w-k-jones Mar 27, 2025
442b984
Fix typing bug for python<3.10
w-k-jones Mar 27, 2025
46a72d6
Fix issues with coordinate interpolation in older xarray versions
w-k-jones Mar 28, 2025
1125c5b
Make use of use_standard_names parameter in coordinate interpolation …
w-k-jones Mar 28, 2025
498acef
Formatting
w-k-jones Mar 28, 2025
5a64f78
Remove duplicate decorators from general_internal utils
w-k-jones Mar 28, 2025
c6c7a31
Add new functions for finding coord names from dataframes and tests
w-k-jones Mar 28, 2025
a34e9e5
Add pd.Series input to find_coord_in_dataframe
w-k-jones Mar 29, 2025
96cf67f
Update calculate_distance to search for coords in dataframe using fin…
w-k-jones Mar 29, 2025
bb97c8e
Formatting
w-k-jones Mar 29, 2025
72c4844
Update idealised case notebooks and check velocity calculation is cor…
w-k-jones Mar 29, 2025
b8ef47e
Update basic example notebooks
w-k-jones Mar 29, 2025
fc08be9
Update test coverage for calculate_distance
w-k-jones Mar 29, 2025
0b1133d
Improve coverage of test_datetime
w-k-jones Mar 29, 2025
fe08335
Improve coverage of test_generators
w-k-jones Mar 29, 2025
158c2e3
Rename and separate internal.basic and internal.general_internal utils
w-k-jones Mar 29, 2025
4586984
Rename and separate tobac.utils.internal.basic and general_internal t…
w-k-jones Mar 29, 2025
a2d25f6
Formatting
w-k-jones Mar 29, 2025
8359a3f
Fix testing error for mismatched coordinates in test_analysis_spatial
w-k-jones Mar 29, 2025
8ae49ce
Add tests for missing error cases in find_dataframe_horizontal_coords
w-k-jones Mar 29, 2025
98b4303
Remove check for error that can never occur
w-k-jones Mar 30, 2025
ee07916
Expand test coverage for errors
w-k-jones Mar 30, 2025
5584ee2
Formatting
w-k-jones Mar 30, 2025
59b76eb
Update model OLR notebook to load/save using xarray and add more visu…
w-k-jones Mar 16, 2025
fd0619a
Update animation_mask_field to use xarray
w-k-jones Mar 26, 2025
6b032c3
Update OLR example with fixed animation
w-k-jones Mar 26, 2025
f3ea0c8
Resolve rebase conflicts
w-k-jones Mar 30, 2025
a88bf13
Fix rebase conflict
w-k-jones Mar 30, 2025
7d1194e
Formatting
w-k-jones Mar 30, 2025
04e2c28
Update model OLR example and add legacy example using iris
w-k-jones Mar 30, 2025
85da012
Fix merge conflicts
w-k-jones Mar 30, 2025
d9959d9
Update internal_utils.coordinates
w-k-jones Mar 30, 2025
a46541f
Revert outdated changes to internal_utils.coordinates
w-k-jones Mar 30, 2025
51c9e29
Remove commented out lines
w-k-jones Mar 30, 2025
b40beb0
Update map_tracks and plot_tracks_mask_field to search for lat/lon co…
w-k-jones Mar 30, 2025
9bbad8c
Update satellite examples notebook
w-k-jones Mar 30, 2025
a1869df
Update precip tracking example
w-k-jones Mar 30, 2025
610da71
Update Track_on_Radar_Segment_on_Satellite example to use xarray, and…
w-k-jones Mar 31, 2025
eb3bfe8
Update remaining notebooks
w-k-jones Mar 31, 2025
935142a
Finish updates to updraft tracking
w-k-jones Mar 31, 2025
6f66e06
Formatting
w-k-jones Mar 31, 2025
f7f9ede
Update bulk statistics notebooks to use same data path as other examples
w-k-jones Apr 1, 2025
c8a6dd6
Fix issue with PBCs in merge_split_MEST where both min/max for hdim_1…
w-k-jones Apr 2, 2025
b2d23ff
Formatting
w-k-jones Apr 2, 2025
ed04521
Update spatial analysis tests to add documentation and improve granul…
w-k-jones Apr 15, 2025
ec91fe5
Add docstrings for datetime utils
w-k-jones Apr 18, 2025
7731e60
Add documentation to datetime tests
w-k-jones Apr 18, 2025
242054d
Formatting
w-k-jones Apr 18, 2025
17ee9eb
Add documentation to generator tests
w-k-jones Apr 18, 2025
d528cbc
Add documentation for test_utils_coordinates
w-k-jones Apr 18, 2025
d231cb3
Formatting
w-k-jones Apr 18, 2025
4de8d3f
Merge pull request #489 from w-k-jones/fix_datetime_output
w-k-jones Apr 22, 2025
afa9a9f
Merge branch 'RC_v1.6.0' of https://github.com/tobac-project/tobac in…
w-k-jones Apr 22, 2025
85f60a7
Update documentation notebooks to use xarray
w-k-jones Apr 29, 2025
fa4baa5
Update data input docs to state that xarray is now the default format
w-k-jones Apr 29, 2025
dbeaf9a
Remove copied attributes from segmentation mask (e.g. units and stand…
w-k-jones Apr 29, 2025
de7f8ce
Formatting
w-k-jones Apr 29, 2025
472683a
Avoid bug in xarray drop_attrs
w-k-jones Apr 29, 2025
ca2f51d
Merge pull request #487 from w-k-jones/xarray_notebook_update
w-k-jones Apr 30, 2025
5e81a81
Update version number to v1.6.0
w-k-jones Apr 30, 2025
b524dfc
Update changelog
w-k-jones Apr 30, 2025
2f21020
Regenerate examples
w-k-jones Apr 30, 2025
07a2f6b
Add Fabian Ziegner to contributors list
w-k-jones Apr 30, 2025
00b4510
Fix typo in changelog
w-k-jones May 1, 2025
7876d8f
Merge pull request #495 from w-k-jones/v1.6_release
w-k-jones May 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,6 @@ build:
python:
install:
- requirements: doc/requirements.txt
sphinx:
# Path to your Sphinx configuration file.
configuration: doc/conf.py
6 changes: 4 additions & 2 deletions .zenodo.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
"affiliation": "University of Oxford",
"orcid": "0000-0001-9786-3723"
},

{
"name": "Kukulies, Julia",
"affiliation": "NSF National Center for Atmospheric Research",
Expand Down Expand Up @@ -66,7 +65,6 @@
"affiliation": "Leibniz Institute for Tropospheric Research, Leipzig (Germany)",
"orcid": "0000-0002-4524-8152"
},

{
"name": "Pfeifer, Nils",
"affiliation": "Leibniz Institute for Tropospheric Research, Leipzig (Germany)",
Expand All @@ -81,6 +79,10 @@
"name": "Zhang, Xin",
"affiliation": "Nanjing University of Information Science & Technology (China)",
"orcid": "0000-0002-1756-6620"
},
{
"name": "Ziegner, Fabian",
"affiliation":"Leibniz Institute for Tropospheric Research, Leipzig (Germany)"
}
],
"license": "BSD-3-Clause",
Expand Down
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,24 @@
### Tobac Changelog

_**Version 1.6.0:**_

**Enhancements for Users**

- MAJOR CHANGE: Feature detection and segmentation have been changed to use xarray DataArrays for representation of spatial fields instead of Iris Cubes. This change will allow for increased flexibility for input data and ease development. Support for Iris Cube input is maintained through a conversion wrapper. [#354](https://github.com/tobac-project/tobac/pull/354) & [#417](https://github.com/tobac-project/tobac/pull/417)
- Some differences in the output of feature detection are present in the output from feature detection: when using xarray, the column names for coordinates will use the variable names, while with iris the columns will use the standard names. This differences reflects the different ways in which coordinates are named in xarray and iris, and can be overridden by the user using the `use_standard_names` parameter. [#489](https://github.com/tobac-project/tobac/pull/489)

**Bug fixes**

- An issue with datetime output from feature detection being converted to `cftime` format erroneously was fixed, and datetime formats should now match the formatting in the input field. Time matching within a tolerance range now works with any combination of datetime formats [#489](https://github.com/tobac-project/tobac/pull/489)

**Documentation**

- Example notebooks have been converted to use `xarray`, and have had minor enhancements to descriptions and visualisations. A notebook showing the legacy approach using Iris Cube input is retained [#487](https://github.com/tobac-project/tobac/pull/487)

**Internal Enhancements**

- `tobac.utils.internal.basic` was restructured into `coordinates` and `label_props` utils, and new `datetime` and `generators` utils were added [#489](https://github.com/tobac-project/tobac/pull/489)

_**Version 1.5.5:**_

**Bug fixes**
Expand Down
2 changes: 1 addition & 1 deletion doc/big_datasets.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Below is a snippet from a larger notebook demonstrating how to run feature detec
)
out_feature_dfs = db.map(
lambda x: tobac.feature_detection_multithreshold(
x.to_iris(), 4000, **parameters_features
x, 4000, **parameters_features
),
b,
).compute()
Expand Down
182 changes: 90 additions & 92 deletions doc/big_datasets_examples/notebooks/parallel_processing_tobac.ipynb

Large diffs are not rendered by default.

Loading
Loading