Auto File Management Part 3: Install data-management tool as command line program #246
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I also added documentation and integrated the tool into the testing framework.
We plan to eventually install the grdata tool as a standalone command line program. Essentially the build-system will perform some substitutions (the CMake build system uses CMake's built-in ``configure_file`` command while the classic build system uses the analogous ``configure_file.py`` script) This commit introduces a few minor tweaks to grdata.py so that it can more easily be consumed by the ``configure_file.py`` script. - The ``configure_file.py`` script, itself, will ultimately require a few more tweaks so that it doesn't report occurences of python's decorator-syntax as errors - However, this commit minimizes the number of required changes
Among other things, we started using picohash and using the functions in os_utils.ch
The file registry is encoded in the autogenerated file_registry.h file that is produced from file_registry.h.in. To get this to work properly for the Makefile build-system, I needed to add a new feature to ``configure_file.py``. In detail: * ``configure_file.py`` already provided the option to replace a variable in a template file with multiple lines of content read from an external file. We assumed that this option would only be used for formatting multiline strings in printf statements. Consequently, the machinery would replace any new-line characters encountered in the external file with the "\n" escape-sequence used in C strings to represent a new-line. * I added simply added the option to ``configure_file.py`` to do the same thing WITHOUT escaping new-line characters.
This commit makes the following changes to the CMake build system: - it now creates the grdata executable and installs it - it exports the program's location as a variable in grackle.pc - it exposes the program's location through a custom CMake command (that custom CMake command will be revisited in future commits)
mabruzzo
added a commit
to mabruzzo/grackle
that referenced
this pull request
Apr 15, 2025
Motivation ---------- At present, we provide several extremely useful resources for teaching Grackle. But, I've long felt that there's initially something of a learning curve. While all of the examples and the ["Interacting with Grackle in Your Simulation Code"](https://grackle.readthedocs.io/en/latest/Interaction.html) section are extremely rich, detailed, and informative, I always feel like they may include a little **too much** information... **Every** usage of Grackle, boils down to the following flow: 1. We have some data about parcels of gas & we want to know something about it (mostly how chemistry/cooling cause it to evolve) 2. We need to encode information about the gas and the physics we care about in a format that Grackle understands.[^1] 3. We package up the data about the gas parcels into a format that Grackle understands (the ``grackle_field_data`` struct). 4. We call the appropriate function. Unfortunately, this flow is not very obvious from the existing resources (i.e. there's just a little too much going on). But, once you understand the flow ("the big picture"), I think our existing resources are extremely useful. Overview -------- This patch tries to bridge this gap. I came up with the shortest and simplest possible example. (I probably could save a line or 2 if I used the global API, but that was a conscious choice). Then, I wrote a tutorial to try to walk the reader through the example chunk by chunk. To help annotate the example, I wrote a really simple sphinx extension. Future Thoughts --------------- Honestly, this is far from perfect, but I think its a good start. Some thoughts for the future: - It would be nice if Grackle could operate with CGS units. The tutorial could then gloss over a number of details. * Since we already use double-precision within Grackle, I think don't think there are any problems with doing this. But clearly, it needs to wait * I think it would also be nice for teaching the pygrackle bindings - It would be better if the example were even shorter: * the `gr_fields` type proposed within grackle-project#271 will help * a number of my other "experimental APIs" will also help * in the 4.0 release, we may want to revisit the default values for the ``use_grackle`` and ``with_radiative_cooling`` parameters (the current defaults are pretty silly). - if we merge in grackle-project#235, grackle-project#237, and grackle-project#246, then we could easily add a section describing how to try out the example without git commands (those PRs are essential for crafting a sample CMake configuration that can automatically install data-files) [^1]: Of course, Grackle is designed to let callers avoid repeating step 2 over and over again.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is the 3rd in the sequence of PRs introducing automatic data-management. (Following #235 and #237). This is the last one in the sequence
To Recap:
python -m pygrackle <args>…). The tool was only useful for testing pygrackle testingThis PR:
bindirectory next to theincludeandlibdirectories of a grackle install)