[IMPROVEMENT] Combine reverse update data and new update data to more cleverly cache

[mgovers]

Relates to #1100

Describe the problem

The PGM splits grid calculations into several mathematical sub-problems, each with their own level of difficulty. For instance:

• Determining the optimal node order from the topology is a very expensive step in the PGM algorithm (especially in the case of meshed grids; see also https://github.com/orgs/PowerGridModel/discussions/24). However, it only needs to happen when the topology actually changes (e.g., switch statuses).
• Less expensive, but still significant, is the construction of the Y_bus from the topology and the electrical parameters. However, it only needs to happen when either the topology and/or the electrical parameters (e.g., tap position) change.
• The actual matrix equation solving, which always needs to happen for every scenario.

To ensure optimal performance, topology and/or Y_bus are cached between scenarios if possible (see also https://power-grid-model.readthedocs.io/en/stable/user_manual/performance-guide.html#topology-caching).

The way to determine whether the topology or the Y_bus can remain cached, is using is_topology_up_to_data_, is_sym_parameter_up_to_date_ and is_asym_parameter_up_to_date_ member variables in MainModelImpl (see

power-grid-model/power_grid_model_c/power_grid_model/include/power_grid_model/main_model_impl.hpp

Lines 357 to 363 in 9d003f5

	void update_state(UpdateChange const& changes) {
	// if topology changed, everything is not up to date
	// if only param changed, set param to not up to date
	is_topology_up_to_date_ = is_topology_up_to_date_ && !changes.topo;
	is_sym_parameter_up_to_date_ = is_sym_parameter_up_to_date_ && !changes.topo && !changes.param;
	is_asym_parameter_up_to_date_ = is_asym_parameter_up_to_date_ && !changes.topo && !changes.param;
	}

). However, currently, the restore_components function (

power-grid-model/power_grid_model_c/power_grid_model/include/power_grid_model/main_model_impl.hpp

Lines 379 to 384 in 9d003f5

	void restore_components(SequenceIdxView const& sequence_idx) {
	(restore_component<ComponentType>(sequence_idx), ...);
	update_state(cached_state_changes_);
	cached_state_changes_ = {};
	}

) invalidates the cache after every scenario if that scenario could no reuse the cache.

Consider, e.g., the input data and update data for a scenario in the batch update for a hypothetical component, as well as its delta with the input data, and the inverse delta, which represents the patch that is required to overlay on the update data to restore the component to its original state as dictated by the input data.

input data	scenario	delta	inverse delta	cache topo?	cache params?
{"status": 1, "tap_pos": 3}	{}	{}	{}	yes	yes
{"status": 1, "tap_pos": 3}	{"status": 1}	{}	{}	yes	yes
{"status": 1, "tap_pos": 3}	{"status": 0}	{"status": 0}	{"status": 1}	no	no
{"status": 1, "tap_pos": 3}	{"tap_pos": 3}	{}	{}	yes	yes
{"status": 1, "tap_pos": 3}	{"tap_pos": 2}	{"tap_pos": 2}	{"tap_pos": 3}	yes	no
{"status": 1, "tap_pos": 3}	{"status": 1, "tap_pos": 2}	"tap_pos": 2}	{"tap_pos": 3}	yes	no
{"status": 1, "tap_pos": 3}	{"status": 0, "tap_pos": 2}	{"status": 0, "tap_pos": 2}	{"status": 1, "tap_pos": 3}	no	no

At first sight, that may seem reasonable. However, if there are two consecutive scenarios in the update data with the same parameters, but that are not cacheable compared to the input data, there is a needless cache invalidation happening between those scenarios.

Improvement proposal

Implement a way to use the cached_inverse_update_ (https://github.com/PowerGridModel/power-grid-model/blob/main/power_grid_model_c/power_grid_model/include/power_grid_model/main_model_impl.hpp#L605) from the previous scenario, together with the next scenario update, to determine the UpdateChange of the combined update.

The implementation should contain the following:

• To store the cache in restore_components:
  • To ensure that the main model instance remains in a clean state after each scenario, the restore_components needs to remain.
  • Instead, the cached_inverse_update.clear(); line can be replaced with something along the lines of std::get<component_index>(previous_scenario_inverse_update_) = std::move(cache_inverse_update);
  • To that end, previous_scenario_inverse_update_ needs to be added as a member variable to the main model.
• In update_component, before the next scenario's update is applied:
  • Use the new scenario's component update, the previous_scenario_inverse_update_ and the input data to determine the combined update_changed (functionality should be added for every component)
  • Immediately after that, previous_scenario_inverse_update_.clear() needs to be called for that component, because it has been invalidated.
• Any exception should clear the previous_scenario_inverse_update_