-
Notifications
You must be signed in to change notification settings - Fork 435
EAMxx: Give option for for tracers to be advected only by dynamics #6789
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
af7fdb2
2df42fe
b1acce2
6bd2c91
f957603
94b7259
c3fb462
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -58,9 +58,7 @@ namespace control { | |
* Note: at this stage, atm procs that act on non-ref grid(s) should be able to create their | ||
* remappers. The AD will *not* take care of remapping inputs/outputs of the process. | ||
* 4) Register all fields and all groups from all atm procs inside the field managers, and proceed | ||
* to allocate fields. Each field manager (there is one FM per grid) will take care of | ||
* accommodating all requests for packing as well as (if possible) bundling of groups. | ||
* For more details, see the documentation in the share/field/field_request.hpp header. | ||
* to allocate fields. For more details, see the documentation in the share/field/field_request.hpp header. | ||
* 5) Set all the fields into the atm procs. Before this point, all the atm procs had were the | ||
* FieldIdentifiers for their input/output fields and FieldGroupInfo for their input/output | ||
* field groups. Now, we pass actual Field and FieldGroup objects to them, where both the | ||
|
@@ -405,7 +403,7 @@ void AtmosphereDriver::reset_accumulated_fields () | |
} | ||
|
||
auto accum_group = m_field_mgr->get_field_group("ACCUMULATED", grid_name); | ||
for (auto f_it : accum_group.m_fields) { | ||
for (auto f_it : accum_group.m_individual_fields) { | ||
auto& track = f_it.second->get_header().get_tracking(); | ||
f_it.second->deep_copy(zero); | ||
track.set_accum_start_time(m_current_ts); | ||
|
@@ -535,8 +533,54 @@ void AtmosphereDriver::create_fields() | |
m_field_mgr = std::make_shared<field_mgr_type>(m_grids_manager); | ||
m_field_mgr->registration_begins(); | ||
|
||
// By now, the processes should have fully built the ids of their | ||
// required/computed fields and groups. Let them register them in the FM | ||
// Before registering fields, check that Field Requests for tracers are compatible | ||
{ | ||
// Create map from tracer name to a vector which contains the field requests for that tracer. | ||
std::map<std::string, std::set<FieldRequest>> tracer_requests; | ||
auto gather_tracer_requests = [&] (FieldRequest req) { | ||
if (not ekat::contains(req.groups, "tracers")) return; | ||
|
||
std::string fname = req.fid.name(); | ||
if (tracer_requests.find(fname) == tracer_requests.end()) { | ||
tracer_requests[fname] = {req}; | ||
} else { | ||
tracer_requests[fname].emplace(req); | ||
} | ||
}; | ||
for (const auto& req : m_atm_process_group->get_required_field_requests()){ | ||
gather_tracer_requests(req); | ||
} | ||
for (const auto& req : m_atm_process_group->get_computed_field_requests()) { | ||
gather_tracer_requests(req); | ||
} | ||
|
||
// Go through the map entry for each tracer and check that every one | ||
// has the same request for turbulence advection. | ||
for (auto fr : tracer_requests) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Say MAM requests field A, specifying that it should not be turb advected. Another process uses field A, so it requests it, but it has no knowledge of what turb advection is, or at least it doesn't care (the choice may be relevant for MAM, but the rest of the atm doesn't care). In this case, wouldn't the two request conflict? Maybe we should turn this into an enum, like enum TracerAdvection {
DynOnly,
DynAndTurb,
Default,
}; (the names are up for debate, ofc). In this case, the default would be Maybe I'm over thinking it, but there is no way, with a bool, to check if some values for turb_advected_tracers are true b/c the customer asked for true, or simply b/c they didn't care, and relied on the default. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The enum may make things complicated, in which case we may also consider whether this check is needed. That is, as long as there is ONE request that says "don't do turb advection", then we don't add it to the turb advected tracers. Maybe I need to better understand the case we are trying to guard ourselves from... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OTOH, using enums in the add_tracer<Required>("blah",units, grid,1,false); vs add_tracer<Required>("blah",units, grid,1,NoTurbAdvection); There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I may be leaning towards getting rid of this check. I think (but will verify) that the changes that evolved in the FM since I wrote this check actually make it a bit redundant? It should error out with inconsistent requests, we just may not get the output about which processes conflict. With regards to the enums, I may be having a change of heart. I've always avoided in the case where a bool was all that's needed, but these signatures definitely are less clear and There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same here: I often though "why adding a new type when a bool flag does it?". But code clarity with enums is on a whole other level. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Follow up: We need this check given how group requests do group logic, namely, if a single processes says "I want A in group G" then A gets added to G, even if some processes intentionally did not add A to G. I think this option is more of "I want to explicitly state that A should not be in group G" and then these checks ensure no one added it. |
||
const auto reqs = fr.second; | ||
|
||
std::set<bool> turb_advect_types; | ||
for (auto req : reqs) { | ||
turb_advect_types.emplace(ekat::contains(req.groups, "turbulence_advected_tracers")); | ||
} | ||
|
||
if (turb_advect_types.size()!=1) { | ||
std::ostringstream ss; | ||
ss << "Error! Incompatible tracer request. Turbulence advection requests not consistent among processes.\n" | ||
" - Tracer name: " + fr.first + "\n" | ||
" - Requests (process name, grid name, is tracers turbulence advected):\n"; | ||
for (auto req : reqs) { | ||
const auto grid_name = req.fid.get_grid_name(); | ||
const bool turb_advect = ekat::contains(req.groups, "turbulence_advected_tracers"); | ||
ss << " - (" + req.calling_process + ", " + grid_name + ", " + (turb_advect ? "true" : "false") + ")\n"; | ||
} | ||
EKAT_ERROR_MSG(ss.str()); | ||
} | ||
} | ||
} | ||
|
||
// Register required/computed fields. By now, the processes should have | ||
// fully built the ids of their required/computed fields and groups | ||
for (const auto& req : m_atm_process_group->get_required_field_requests()) { | ||
m_field_mgr->register_field(req); | ||
} | ||
|
@@ -599,8 +643,8 @@ void AtmosphereDriver::create_fields() | |
m_field_mgr->add_to_group(fid, "RESTART"); | ||
} | ||
for (const auto& g : m_atm_process_group->get_groups_in()) { | ||
if (g.m_bundle) { | ||
m_field_mgr->add_to_group(g.m_bundle->get_header().get_identifier(), "RESTART"); | ||
if (g.m_monolithic_field) { | ||
m_field_mgr->add_to_group(g.m_monolithic_field->get_header().get_identifier(), "RESTART"); | ||
} else { | ||
for (const auto& fn : g.m_info->m_fields_names) { | ||
m_field_mgr->add_to_group(fn, g.grid_name(), "RESTART"); | ||
|
@@ -1128,19 +1172,19 @@ void AtmosphereDriver::set_initial_conditions () | |
// ...then the input groups | ||
m_atm_logger->debug(" [EAMxx] Processing input groups ..."); | ||
for (const auto& g : m_atm_process_group->get_groups_in()) { | ||
if (g.m_bundle) { | ||
process_ic_field(*g.m_bundle); | ||
if (g.m_monolithic_field) { | ||
process_ic_field(*g.m_monolithic_field); | ||
} | ||
for (auto it : g.m_fields) { | ||
for (auto it : g.m_individual_fields) { | ||
process_ic_field(*it.second); | ||
} | ||
} | ||
m_atm_logger->debug(" [EAMxx] Processing input groups ... done!"); | ||
|
||
// Some fields might be the subfield of a group's bundled field. In that case, | ||
// we only need to init one: either the bundled field, or all the individual subfields. | ||
// Some fields might be the subfield of a group's monolithic field. In that case, | ||
// we only need to init one: either the monolithic field, or all the individual subfields. | ||
// So loop over the fields that appear to require loading from file, and remove | ||
// them from the list if they are the subfield of a bundled field already inited | ||
// them from the list if they are the subfield of a groups monolithic field already inited | ||
// (perhaps via initialize_constant_field, or copied from another field). | ||
for (auto& it1 : ic_fields_names) { | ||
const auto& grid_name = it1.first; | ||
|
@@ -1244,17 +1288,17 @@ void AtmosphereDriver::set_initial_conditions () | |
} | ||
m_atm_logger->debug(" [EAMxx] Processing fields to copy ... done!"); | ||
|
||
// It is possible to have a bundled group G1=(f1,f2,f3), | ||
// It is possible to have a monolithically allocated group G1=(f1,f2,f3), | ||
// where the IC are read from file for f1, f2, and f3. In that case, | ||
// the time stamp for the bundled G1 has not be inited, but the data | ||
// the time stamp for the monolithic field of G1 has not be inited, but the data | ||
// is valid (all entries have been inited). Let's fix that. | ||
m_atm_logger->debug(" [EAMxx] Processing subfields ..."); | ||
for (const auto& g : m_atm_process_group->get_groups_in()) { | ||
if (g.m_bundle) { | ||
auto& track = g.m_bundle->get_header().get_tracking(); | ||
if (g.m_monolithic_field) { | ||
auto& track = g.m_monolithic_field->get_header().get_tracking(); | ||
if (not track.get_time_stamp().is_valid()) { | ||
// The bundled field has not been inited. Check if all the subfields | ||
// have been inited. If so, init the timestamp of the bundled field too. | ||
// The groups monolithic field has not been inited. Check if all the subfields | ||
// have been inited. If so, init the timestamp of the monlithic field too. | ||
const auto& children = track.get_children(); | ||
bool all_inited = children.size()>0; // If no children, then something is off, so mark as not good | ||
for (auto wp : children) { | ||
|
@@ -1632,7 +1676,7 @@ void AtmosphereDriver::run (const int dt) { | |
} | ||
|
||
auto rescale_group = m_field_mgr->get_field_group("DIVIDE_BY_DT", gname); | ||
for (auto f_it : rescale_group.m_fields) { | ||
for (auto f_it : rescale_group.m_individual_fields) { | ||
f_it.second->scale(Real(1) / dt); | ||
} | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this check is pointless. The else branch should work regardless, no?