[FEATURE] Add utility function to reorganize scenarios according to the performance guide

[mgovers]

Describe the problem

Different use cases require different sets of calculations. Since the PGM treats scenarios in an isolated fashion, the core is agnostic to the order of calculations.

However, some caching may happen in the PGM core to speed up the calculations. But whether and what can be cached depends on the order and contents of the scenarios in the update data, as well as amount of threads used during the calculation.

Guidelines to speed up the calculation are documented in https://power-grid-model.readthedocs.io/en/stable/user_manual/performance-guide.html#batch-calculations .

For many use cases, the order of scenarios does not really matter to the user (e.g., when doing N-1 simulations). Even when the order does matter (e.g., when doing time series calculations), only the final order matters.

Improvement proposal

• Add a function reorder_scenarios_for_performance(*, BatchDataset update_data) -> tuple[BatchDataset, list[int]] to the power_grid_model.utils module that:
  • creates an ordering reordering: list[int] of the scenarios (following the conventions of np.argsort) in the update data according to the performance guidelines, that is:
    • it keeps scenarios with the same topology (e.g., same switch statuses) together
    • within the same topology, it keeps scenarios with the same electrical parameters (e.g., branch parameters, tap position, ...) together
  • creates a copy (!!!) of the update data with the reordered order of scenarios, such that it is equivalent to

    for reordered_idx, scenario_idx in enumerate(reordering):
        np.testing.assert_array_equal(
            get_dataset_scenario(reordered_update_data, reordered_idx),
            get_dataset_scenario(update_data, scenario_idx)
        )
    # for row-based dense batch data, this is equivalent to:
    for component in update_data:
        np.testing.assert_array_equal(
            reordered_update_data[component][:],
            update_data[component][reordering]
        )

  • computes the inverse_ordering: list[int] of the scenarios, such that

    for scenario_idx, reordered_idx in enumerate(inverse_ordering):
        np.testing.assert_array_equal(
            get_dataset_scenario(reordered_update_data, reordered_idx),
            get_dataset_scenario(update_data, scenario_idx),
        )
    # for row-based dense batch data, this is equivalent to:
    for component in update_data:
        np.testing.assert_array_equal(
            reordered_update_data[component][:],
            update_data[component][inverse_ordering],
        )

  • returns both the reordered_update_data and the inverse_ordering
  • works for both row-based, columnar data, dense and sparse data, as well as a mixture of the two
  • has a clear docstring
• Add tests for row-based, columnar, dense, sparse and mixed data.