-
Notifications
You must be signed in to change notification settings - Fork 435
EAMxx: Implement field aliasing feature with := syntax and metadata support #7570
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
62097eb
Initial plan
Copilot 96bc2bb
Implement field aliasing functionality for EAMxx I/O operations
Copilot 5b2a422
Add tests and documentation for field aliasing feature
Copilot ef4461c
EAMxx: Implement field aliasing feature for netcdf I/O operations
Copilot 48f3787
Fix documentation, field names, and add integration test structure
Copilot a8f50de
Improve string handling and enhance integration tests
Copilot 5e05b89
EAMxx: fixes to copilot impl
mahf708 9e4fbc8
EAMxx: update docs
mahf708 fdd2652
EAMxx: Add alias metadata and change delimiter to :=
Copilot 6f73809
EAMxx: Fix data interpolation issue by removing unused e2str call
Copilot 1a348d8
EAMxx: alias only outputs, fix docs
mahf708 d891e9c
EAMxx: add an additional check for multiple ":="
mahf708 bb728c1
EAMxx: fix ekat compatibility
mahf708 121d730
EAMxx: beef up test
mahf708 836aae0
EAMxx: slim+format io_alias test
mahf708 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,83 @@ | ||
# EAMxx Field Aliasing Feature | ||
|
||
This document demonstrates the field aliasing feature for EAMxx I/O operations. | ||
|
||
## Overview | ||
|
||
The field aliasing feature allows users to specify custom names for | ||
variables in netcdf output files while maintaining the original | ||
internal field names in the model. This is useful for: | ||
|
||
- Creating shorter, more convenient variable names for output | ||
- Maintaining compatibility with external tools that expect specific variable names | ||
- Providing user-friendly names for complex internal field names | ||
|
||
## Syntax | ||
|
||
The alias syntax uses the delimiter `:=` to separate the alias name | ||
from the internal field name: | ||
|
||
```yaml | ||
alias_name:=internal_field_name | ||
``` | ||
|
||
## YAML Configuration Examples | ||
|
||
### Basic Usage | ||
|
||
```yaml | ||
field_names: | ||
- "LWP:=LiqWaterPath" # Alias LWP for LiqWaterPath | ||
- "SWP:=SolidWaterPath" # Alias SWP for SolidWaterPath | ||
- "T:=T_mid" # Alias T for T_mid | ||
- "qv" # Regular field name (no alias) | ||
``` | ||
|
||
### Mixed Usage | ||
|
||
You can mix aliased and non-aliased fields in the same configuration: | ||
|
||
```yaml | ||
field_names: | ||
- "T_mid" # Regular field name | ||
- "LWP:=LiqWaterPath" # Aliased field | ||
- "p_mid" # Regular field name | ||
- "RH:=RelativeHumidity" # Aliased field | ||
``` | ||
|
||
## Output Behavior | ||
|
||
When using aliases: | ||
|
||
1. **NetCDF Variables**: The netcdf file will contain variables | ||
named according to the aliases | ||
|
||
- `LWP` instead of `LiqWaterPath` | ||
- `T` instead of `T_mid` | ||
- `RH` instead of `RelativeHumidity` | ||
|
||
2. **Internal Processing**: All internal model operations use the | ||
original field names | ||
|
||
- Field validation uses `LiqWaterPath`, `T_mid`, etc. | ||
- Diagnostic calculations use original names | ||
- Memory management uses original field structures | ||
|
||
3. **Metadata**: Variable attributes (units, long_name, etc.) | ||
are preserved from the original fields, and `eamxx_name` | ||
is added to the netcdf files to document aliasing | ||
|
||
## Caveats | ||
|
||
Currently, a field can be requested only once in a single stream, | ||
and either the original name or the alias name counts. | ||
|
||
```yaml | ||
field_names: | ||
- "LWP:=" # Error: empty field name | ||
- ":=LiqWaterPath" # Error: empty alias name | ||
- "LWP:=Field1" # OK | ||
- "LWP:=Field2" # Error: duplicate alias LWP | ||
- "LWP1:=LiqWaterPath" # OK | ||
- "LWP2:=LiqWaterPath" # Error: duplicate field LiqWaterPath | ||
``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The first part of this fcn could be replaced with
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what if field_spec has no
:=
?Should I just add this check at the end?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ekat::split
would simply put the whole input string intokens[0]
, so the suggestion above may help with that too.Edit: nvm, my code is wrong. :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, if tokens.size()==1, we should just return
{ekat::trim(tokens[0]), ekat::trim(tokens[0]})
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, if there's a second
:=
in the string, it would fail with not found in the list of fields/diags. I think that's good enough to handle that error. We can improve on that later.Anything else you see blocking this PR?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the version I posted in my comment, it should throw an exception since
tokens.size()>2
.Edit: so, yeah, currently it would fail since the RHS would be not found as a diag.