Implement solver support for SPECFEM++

[bch0w]

SPECFEM++ developer @Rohit-Kakodkar and I agreed it would be great for the community to build in SeisFlows support for SPECFEM++, to handle workflow management for forward and adjoint simulations (and eventually inversions!).

SPECFEM++ works slightly differently than the Fortran version of the code (e.g., YAML parameter file, different source setups) and will likely need an entirely new Solver base class. Likely some minor reworking of the SeisFlows archicture will be needed as well to accommodate these changes. This will be a great first step into incorporating other numerical solvers into the SeisFlows codebase, which until now has only worked with different flavors of Fortran-based SPECFEM.

Putting this issue here as a tracker for progress on this front.

[bch0w]

Hi @Rohit-Kakodkarm here is a broad-strokes outline of what I think needs to be done to implement SPECFEM++ in SeisFlows. I'll use the terms 'specfemft' to refer to the Fortran version, and 'specfempp' to refer to SPECFEM++.

Our first goal should be to run a forward simulation (https://github.com/adjtomo/seisflows/blob/devel/seisflows/workflow/forward.py) with SeisFlows controlling SPECFEM++ under the hood. From there we can build to adjoint simulations, etc.

In the main workflow classes of SeisFlows, specfemft (2d, 3d and 3d_globe) is generalized as solver so we will need to make sure that a newly written specfempp solver class contains all the same generic functions that are expected of the solver class.

1. Solver class: We'll need to follow some of the structure laid out in the current specfemft version (https://github.com/adjtomo/seisflows/blob/devel/seisflows/solver/specfem.py)

• This Class is responsible for wrapping executables, and explicitly defining the structure (directories, files) of a specfem working directory for internal consistency
• Main functions are forward_simulation and adjoint_simulation which perform set up and tear down tasks (e.g., setting correct parameters and saving output files) as well as running specfemft executable
• We'll need to have generic functions/attributes linked to solver in the Forward workflow (https://github.com/adjtomo/seisflows/blob/devel/seisflows/workflow/forward.py)
• These include: check_model_values, source_name, cwd, import_model, forward_simulation
• I started a boilerplate class started here: https://github.com/adjtomo/seisflows/blob/feature-specfempp/seisflows/solver/specfempp.py

3. Tool module: specfemft has a number of helper functions for e.g., navigating a specfemft working directory, getting/setting parameter file parameters (https://github.com/adjtomo/seisflows/blob/devel/seisflows/tools/specfem.py).

• For simplicity let's define these as functions within the specfempp solver module, and if that gets too large we can split them off into a separate tool module
  -getpar and setpar probably the most important, allows other modules to get and set parameters in a SPECFEM parameter file
• This is probably where we'll want to implement the functionality to automatically generate the necessary input files (parameter file, source file)

4. Model class: SeisFlows currently works with specfemft Fortran binary models by interacting with them through the Model class, which stores an internal NumPy representation of a distributed model.

• This is mainly used for Inversion workflows when models need to be updated
• For forward and adjoint simulations only, this will not be important and can hopefully be sidelined
• In the future we'll have to think about how to implement a specfempp Model class that behaves the same

5. Examples: Once we get the core functionality working, we can think about setting up an example following something like: https://github.com/adjtomo/seisflows/blob/feature-specfempp/seisflows/examples/sfexample2d.py