Skip to content

Conversation

chillenzer
Copy link
Contributor

This PR adds rudimentary support for output via the openPMD plugin.

Available capabilities:

  • ParticleDump(species) with native species
  • FieldDump(fieldname) with native fields
  • OpenPMDConfig except for range (and backend_config untested)

Missing capabilities:

  • DerivedFieldDump from particles
  • RangeSpec for modelling ranges
  • Filter
  • Potentially convenience interfaces like species_all, fields_all.

Some explanations:

  • In discussion with @franzpoeschel we decided that the best implementation is to generate a .toml file for each instance of the openPMD plugin that PIConGPU is supposed to use. This is straightforward to achieve but is not quite aligned with the pypicongpu approach to rendering:
    • In pypicongpu, the runner is responsible for handling the creation of a hardcoded (!) set of files in a specific location. The individual components are only supposed to provide some content to fill in.
    • With the current implementation, the individual component is allowed to create an arbitrary set of files. I have hacked in a way to spread the information about the location where to put them. It's not particularly nice but the alternative ain't pretty either. Open for discussions here.
  • In above approach, most information circumvents the pypicongpu.json by being directly located inside of the .toml files. This means a two-step approach would be necessary to get the full information from the pypicongpu.json: 1. Parse pypicongpu.json and 2. extract the corresponding .toml file's name and parse that as well. If parsing pypicongpu.json is relevant to your workflow, there are two options:
    • My current sketches for database integration, etc., have a reporting capability built-in at a later stage in which the plugins explicitly report which output is to be expected where. As I provide tooling to easily extract that information, no parsing is necessary at all and we are free to adjust the internal structure.
    • We could add redundant information to the pypicongpu.json file that's not necessary for rendering the templates but just meant to inform the user after the fact. I typically don't fancy this kind of solution.
  • The user interface in PIConGPU and PICMI differs: PIConGPU has a technical perspective in which the user explicitly instantiates plugins inside of PIConGPU while PICMI has a more semantic perspective in which the user describes what physical output they want to see. Translation from one to the other is done by picmi/simulation.py which gathers all diagnostics getting dumped into the same file(s) into one instance of the openPMD plugin.
  • I have written another end-to-end test for testing this feature. As they are not yet run in CI, please be so kind to run them upon review. I'll eventually take the time to figure out how to fix running them in CI, I promise!

CAUTION: The handling of ranges is still inconsistent (because it's not yet implemented). Should you manage to somehow pass a range through the different layers (not sure if anything's hindering you), an OpenPMDConfig instance that's identical up to range would be treated as different. This implies that there would be two OpenPMDWriter instances in PIConGPU writing to the same file. I'll take care of this once range is properly fleshed out.

@chillenzer chillenzer added this to the 0.9.0 / next stable milestone Oct 14, 2025
@chillenzer chillenzer added CI:no-compile CI is skipping compile/runtime tests but runs PICMI tests PICMI pypicongpu and picmi related labels Oct 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI:no-compile CI is skipping compile/runtime tests but runs PICMI tests PICMI pypicongpu and picmi related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant