-
Notifications
You must be signed in to change notification settings - Fork 309
Refactor resampler module to submodules #3124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
pnuu
wants to merge
34
commits into
pytroll:main
Choose a base branch
from
pnuu:refactor-resample
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,537
−1,245
Open
Changes from 27 commits
Commits
Show all changes
34 commits
Select commit
Hold shift + click to select a range
e873d3b
Refactor resampler module to submodules
lahtinep b65098a
Update resample imports
lahtinep f3712ab
Update and move resampling docstring
lahtinep 0b59ca8
Fix DOS line-endings, fix get_area_def import
lahtinep 9051519
Fix imports to use new resampler modules
lahtinep 41f5ef0
Simplify prepare_resampler()
lahtinep 43f4b33
Simplify add_xy_coords()
lahtinep 80b16e1
Simplify BucketResamplerBase.resample()
lahtinep b16e834
Simplify KDTreeResampler.load_neighbour_info()
lahtinep d68afb5
Simplify NativeResampler.compute()
lahtinep ea5d24b
Simplify native._rechunk_if_nonfactor_chunks()
lahtinep c71f33d
Simplify _check_crs_units and add_xy_coords() more
lahtinep 86a4def
Reduce cyclomatic complecity of NativeResampler._expand_reduce()
lahtinep 44ee39b
Make base.hash_dict a private function
lahtinep e33ea4b
Make get_area_file and update_resampled_coords private functions
lahtinep 412d848
Fix import
lahtinep d7e4e9e
Move get_area_def and get_area_file to a new module satpy.area_utils
lahtinep 125e691
Create satpy.coords_utils module
lahtinep f24507f
Fix get_area_def patching
lahtinep 28e02ca
Merge branch 'main' into refactor-resample
lahtinep 9371f45
Fix bucket doc references, again
lahtinep 5a73b6e
Merge branch 'main' into refactor-resample
pnuu e5a4db7
Use helper function instead of if/elif/else in satpy.resample imports
pnuu 3fcbe6d
Fix resample imports and doc references
pnuu 5fe8f06
Remove unnecessary pyproj<2.0 check
pnuu ea17ec0
Merge branch 'main' into refactor-resample
pnuu bfdb334
Fix resampling related invalid sphinx references
djhoese 9d5a3a7
Move BaseResampler import from top level to functions
pnuu 9f9a385
Refactor collection of resampler classes
pnuu 561101b
Merge branch 'refactor-resample' of https://github.com/pnuu/satpy int…
pnuu 5bbd4f1
Rename area_utils.py to area.py
pnuu 0b2be45
Rename coords_utils.py to coords.py
pnuu 7ff4f6d
Move resample_dataset and prepare_rempler import to functions
pnuu 80296ee
Add a test for warnings issued when importing directly from satpy.res…
pnuu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
#!/usr/bin/env python | ||
# Copyright (c) 2015-2025 Satpy developers | ||
# | ||
# This file is part of satpy. | ||
# | ||
# satpy is free software: you can redistribute it and/or modify it under the | ||
# terms of the GNU General Public License as published by the Free Software | ||
# Foundation, either version 3 of the License, or (at your option) any later | ||
# version. | ||
# | ||
# satpy is distributed in the hope that it will be useful, but WITHOUT ANY | ||
# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR | ||
# A PARTICULAR PURPOSE. See the GNU General Public License for more details. | ||
# | ||
# You should have received a copy of the GNU General Public License along with | ||
# satpy. If not, see <http://www.gnu.org/licenses/>. | ||
"""Utility functions for area definitions.""" | ||
from ._config import config_search_paths, get_config_path | ||
|
||
|
||
def get_area_file(): | ||
"""Find area file(s) to use. | ||
|
||
The files are to be named `areas.yaml` or `areas.def`. | ||
""" | ||
paths = config_search_paths("areas.yaml") | ||
if paths: | ||
return paths | ||
else: | ||
return get_config_path("areas.def") | ||
|
||
|
||
def get_area_def(area_name): | ||
"""Get the definition of *area_name* from file. | ||
|
||
The file is defined to use is to be placed in the $SATPY_CONFIG_PATH | ||
directory, and its name is defined in satpy's configuration file. | ||
""" | ||
try: | ||
from pyresample import parse_area_file | ||
except ImportError: | ||
from pyresample.utils import parse_area_file | ||
return parse_area_file(get_area_file(), area_name)[0] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,131 @@ | ||
#!/usr/bin/env python | ||
# Copyright (c) 2015-2025 Satpy developers | ||
# | ||
# This file is part of satpy. | ||
# | ||
# satpy is free software: you can redistribute it and/or modify it under the | ||
# terms of the GNU General Public License as published by the Free Software | ||
# Foundation, either version 3 of the License, or (at your option) any later | ||
# version. | ||
# | ||
# satpy is distributed in the hope that it will be useful, but WITHOUT ANY | ||
# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR | ||
# A PARTICULAR PURPOSE. See the GNU General Public License for more details. | ||
# | ||
# You should have received a copy of the GNU General Public License along with | ||
# satpy. If not, see <http://www.gnu.org/licenses/>. | ||
"""Utility functions for coordinates.""" | ||
|
||
import logging | ||
|
||
import xarray as xr | ||
|
||
LOG = logging.getLogger(__name__) | ||
|
||
|
||
def add_xy_coords(data_arr, area, crs=None): | ||
"""Assign x/y coordinates to DataArray from provided area. | ||
|
||
If 'x' and 'y' coordinates already exist then they will not be added. | ||
|
||
Args: | ||
data_arr (xarray.DataArray): data object to add x/y coordinates to | ||
area (pyresample.geometry.AreaDefinition): area providing the | ||
coordinate data. | ||
crs (pyproj.crs.CRS or None): CRS providing additional information | ||
about the area's coordinate reference system if available. | ||
Requires pyproj 2.0+. | ||
|
||
Returns (xarray.DataArray): Updated DataArray object | ||
|
||
""" | ||
if "x" in data_arr.coords and "y" in data_arr.coords: | ||
# x/y coords already provided | ||
return data_arr | ||
if "x" not in data_arr.dims or "y" not in data_arr.dims: | ||
# no defined x and y dimensions | ||
return data_arr | ||
if not hasattr(area, "get_proj_vectors"): | ||
return data_arr | ||
x, y = area.get_proj_vectors() | ||
|
||
# convert to DataArrays | ||
y_attrs = {} | ||
x_attrs = {} | ||
|
||
_check_crs_units(crs, x_attrs, y_attrs) | ||
|
||
y = xr.DataArray(y, dims=("y",), attrs=y_attrs) | ||
x = xr.DataArray(x, dims=("x",), attrs=x_attrs) | ||
return data_arr.assign_coords(y=y, x=x) | ||
|
||
|
||
def _check_crs_units(crs, x_attrs, y_attrs): | ||
if crs is None: | ||
return | ||
units = crs.axis_info[0].unit_name | ||
# fix udunits/CF standard units | ||
units = units.replace("metre", "meter") | ||
if units == "degree": | ||
y_attrs["units"] = "degrees_north" | ||
x_attrs["units"] = "degrees_east" | ||
else: | ||
y_attrs["units"] = units | ||
x_attrs["units"] = units | ||
|
||
|
||
def add_crs_xy_coords(data_arr, area): | ||
"""Add :class:`pyproj.crs.CRS` and x/y or lons/lats to coordinates. | ||
|
||
For SwathDefinition or GridDefinition areas this will add a | ||
`crs` coordinate and coordinates for the 2D arrays of `lons` and `lats`. | ||
|
||
For AreaDefinition areas this will add a `crs` coordinate and the | ||
1-dimensional `x` and `y` coordinate variables. | ||
|
||
Args: | ||
data_arr (xarray.DataArray): DataArray to add the 'crs' | ||
coordinate. | ||
area (pyresample.geometry.AreaDefinition): Area to get CRS | ||
information from. | ||
|
||
""" | ||
from pyresample.geometry import SwathDefinition | ||
|
||
crs, data_arr = _add_crs(area, data_arr) | ||
|
||
# Add x/y coordinates if possible | ||
if isinstance(area, SwathDefinition): | ||
# add lon/lat arrays for swath definitions | ||
# SwathDefinitions created by Satpy should be assigning DataArray | ||
# objects as the lons/lats attributes so use those directly to | ||
# maintain original .attrs metadata (instead of converting to dask | ||
# array). | ||
lons = area.lons | ||
lats = area.lats | ||
lons.attrs.setdefault("standard_name", "longitude") | ||
lons.attrs.setdefault("long_name", "longitude") | ||
lons.attrs.setdefault("units", "degrees_east") | ||
lats.attrs.setdefault("standard_name", "latitude") | ||
lats.attrs.setdefault("long_name", "latitude") | ||
lats.attrs.setdefault("units", "degrees_north") | ||
# See https://github.com/pydata/xarray/issues/3068 | ||
# data_arr = data_arr.assign_coords(longitude=lons, latitude=lats) | ||
else: | ||
# Gridded data (AreaDefinition/StackedAreaDefinition) | ||
data_arr = add_xy_coords(data_arr, area, crs=crs) | ||
return data_arr | ||
|
||
|
||
def _add_crs(area, data_arr): | ||
from pyproj import CRS | ||
|
||
if hasattr(area, "crs"): | ||
crs = area.crs | ||
else: | ||
# default lat/lon projection | ||
latlon_proj = "+proj=latlong +datum=WGS84 +ellps=WGS84" | ||
proj_str = getattr(area, "proj_str", latlon_proj) | ||
crs = CRS.from_string(proj_str) | ||
data_arr = data_arr.assign_coords(crs=crs) | ||
return crs, data_arr |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.