Building on Perlmutter

[egstern]

I'm a newbie on building ImpactX but I've been building other software at NERSC for a long time. I get suspicious when I'm told to just run a bunch of scripts without knowing what they are doing so I was looking at the scripts mentioned in the instructions here:
https://impactx.readthedocs.io/en/latest/install/hpc/perlmutter.html

First thing that bothers me is that you are expected to put the source in $HOME/src/impactx. Maybe I have other things in $HOME/src and don't want to mix them up with impactx, or maybe I want to have multiple live impactx branches going at once. How about putting it in $IX_HOME/src/impactx and the instructions are to set IX_HOME to an appropriate value?

The profile script has a bunch of old stuff hardcoded that does not even exist anymore. The named directory does not exist.

# optional: for QED support with detailed tables
export BOOST_ROOT=/global/common/software/spackecp/perlmutter/e4s-23.05/default/spack/opt/spack/linux-sles15-zen3/gcc-11.2.0/boost-1.82.0-ow5r5qrgslcwu33grygouajmuluzuzv3

The build procedure seems to put the build products in a specific directory in CFS. That should also be easily changeable. I might want or need to have several built versions accessible at once.

The script also loads various modules with specific version numbers. Module version numbers change frequently. In fact, the version currently listed is not the default version. Why? There should be a comment next to it about that.

[egstern]

The dependency build script $HOME/src/impactx/docs/source/install/hpc/perlmutter-nersc/install_gpu_dependencies.sh also references files in the impactx source tree so if that location changes it has to change there as well.

+ python3 -m pip install --upgrade -r /global/homes/e/egstern/src/impactx/requirements.txt
ERROR: Could not open requirements file: [Errno 2] No such file or directory: '/global/homes/e/egstern/src/impactx/requirements.txt'

[ax3l]

Oh that is right, I forgot about this in my initial response.

Yes, we should parametrize the script with a SRC_DIR and SW_DIR variable as we usually do to make it flexible for people that want to change this.

[ax3l]

Thank you for looking through the instructions and sharing your feedback!

I get suspicious when I'm told to just run a bunch of scripts without knowing what they are doing so I was looking at the scripts mentioned in the instructions here:
https://impactx.readthedocs.io/en/latest/install/hpc/perlmutter.html

That's great that you look into those. The online documentation does intentionally have a collapse/expand feature to let you inspect those and they are part of the source directory in docs/source/install/hpc/

First thing that bothers me is that you are expected to put the source in $HOME/src/impactx

That is fully optional. The scripts do not require that you put the source of ImpactX there and you can indeed use whatever location you like, have multiple copies, etc. As an advanced user, you can adjust this part as you need. We just make it explicit in the docs here to make it simple for less experienced developers/users and to make sure that if they copy-paste the instructions top-to-bottom that they will have a working instruction set.

Update: oh wait, Python install logic reads this. See here.

BOOST_ROOT

We do not need this. Copied over from WarpX instructions, please submit a PR to clean this line out.

The build procedure seems to put the build products in a specific directory in CFS. That should also be easily changeable. I might want or need to have several built versions accessible at once.

Yes, we usually use a SRC_DIR and SW_DIR variable in our scripts in ImpactX/WarpX that can be adjusted for all dependencies. Maybe the one in ImpactX needs that overhaul and is outdated, we should def use this.

The script also loads various modules with specific version numbers. Module version numbers change frequently. In fact, the version currently listed is not the default version. Why? There should be a comment next to it about that.

Fixing version numbers is fully intentional. There is zero guarantee that two version numbers provided in modules are ABI compatible. To avoid nasty bugs at runtime, we rather throw an error if a version is gone than letting it corrupt a production run that was compiled with an older version, then the default changed, and at runtime the newer default is taken.
Explicit updates help here, that's why we do the extra work to maintain (document) tested versions of modules.

[egstern]

Hi Axel, Could you walk me through the preferred workflow for ImpactX PRs? Could you point me to the right place if I'm going to edit the documentation directly? Thanks, Eric

…

On Tue, Jun 10, 2025 at 5:59 PM Axel Huebl ***@***.***> wrote: Thank you for looking through the instructions and sharing your feedback! I get suspicious when I'm told to just run a bunch of scripts without knowing what they are doing so I was looking at the scripts mentioned in the instructions here: https://impactx.readthedocs.io/en/latest/install/hpc/perlmutter.html That's great that you look into those. The online documentation does intentionally have a collapse/expand feature to let you inspect those and they are part of the source directory in docs/source/install/hpc/ First thing that bothers me is that you are expected to put the source in $HOME/src/impactx That is fully optional. The scripts do not require that you put the source of ImpactX there and you can indeed use whatever location you like, have multiple copies, etc. As an advanced user, you can adjust this part as you need. We just make it explicit in the docs here to make it simple for less experienced developers/users and to make sure that if they copy-paste the instructions top-to-bottom that they will have a working instruction set. BOOST_ROOT We do not need this. Copied over from WarpX instructions, please submit a PR to clean this line out. The build procedure seems to put the build products in a specific directory in CFS. That should also be easily changeable. I might want or need to have several built versions accessible at once. Yes, we usually use a SRC_DIR and SW_DIR variable in our scripts in ImpactX/WarpX that can be adjusted for all dependencies. Maybe the one in ImpactX needs that overhaul and is outdated, we should def use this. The script also loads various modules with specific version numbers. Module version numbers change frequently. In fact, the version currently listed is not the default version. Why? There should be a comment next to it about that. Fixing version numbers is fully intentional. There is zero guarantee that two version numbers provided in modules are ABI compatible. To avoid nasty bugs at runtime, we rather throw an error if a version is gone than letting it corrupt a production run that was compiled with an older version, then the default changed, and at runtime the newer default is taken. Explicit updates help here, that's why we do the extra work to maintain (document) tested versions of modules. — Reply to this email directly, view it on GitHub <#978 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD4V5DARBS7EEOK3YI2A7ZD3C5PN3AVCNFSM6AAAAAB6DQEMWWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGNBSGY2TMMY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[ax3l]

Hi @egstern,

Back at work today :)
Yes, happy to guide you! Our ImpactX contribution guide is outlined here:
https://impactx.readthedocs.io/en/latest/developers/contributing.html

In essence it is the usual GitHub workflow:

• fork the repository
• start a new branch from the latest development commit for each change: use always a new branch started from development for independent changes
• when done with a branch, push it to your fork and open a pull request (PR) to our mainline repo
  • while we review your pull request and suggest changes, updating (committing) to the branch of a PR will update the PR

The first link is way more detailed and if you like more guidance I am happy to hop on a zoom with you any time! :)

For the documentation in particular, the files are located in docs/source/. Once you submit the file changes for PR there will be a preview build of the docs for you in the PR as part of the tests, but you can also build the docs locally if you prefer that (requirements file).

Best,
Axel