Skip to content

Commit 88dc11e

Browse files
bartgolbartgol
authored
Merge branch 'mjs/eamxx/docs-dev-guide-update' (PR #7163)
Adds TPLs page to Developer Guide and reorganizes Dev Quick-start. [BFB]
2 parents ef7d386 + af658cf commit 88dc11e

File tree

4 files changed

+227
-20
lines changed

4 files changed

+227
-20
lines changed

‎components/eamxx/docs/developer/TPLs.md‎

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
# Third-party Libraries Required by EAMxx
2+
3+
- First, we note that if you are running EAMxx on a supported machine,
4+
then you can expect that the third-party libraries (TPLs) are
5+
installed, properly configured, and recognized by the EAMxx build system.
6+
- However, if you choose to run and/or develop EAMxx on a local machine,
7+
workstation, or unsupported cluster, then you will likely need to build some
8+
number of the TPLs manually.
9+
- **Note:** The exception to this is if you are running on a machine with
10+
a module system that already provides them--e.g., research institutions
11+
that run E3SM regularly.
12+
13+
## List of TPLs
14+
15+
- **Note:** For this section, we will assume you plan to build EAMxx on a fresh
16+
Linux[^only-linux] system with the standard development tools available.
17+
- E.g., a package manager (`apt`, `yum`), `git`, `make`, `cmake`, etc.
18+
19+
??? Abstract "TPLs You Will (almost certainly) Need to Install or Build Manually"
20+
21+
- [OpenMPI](https://docs.open-mpi.org/en/v5.0.x/installing-open-mpi/quickstart.html)
22+
- Only required for builds employing multi-node parallelism.
23+
- Required for parallel configurations of the netCDF libraries.
24+
- Likely to be provided by your package manager or module system.
25+
- [HDF5](https://www.hdfgroup.org/download-hdf5/)
26+
- High-performance data-wrangling library.
27+
- The link above provides `.deb` and `.rpm` binary packages, though
28+
the page also provides a link to build from source.
29+
- [netCDF-C](https://github.com/Unidata/netcdf-c)
30+
- Used for I/O of simulation data in a format containing descriptive metadata.
31+
- [netCDF-fortran](https://github.com/Unidata/netcdf-fortran)
32+
- [PnetCDF](https://parallel-netcdf.github.io)
33+
- Only required for builds employing multi-node parallelism.
34+
35+
**Build Tips**
36+
37+
- As of 2024, the author found much less resistance configuring the build
38+
for the netCDF libraries using Autoconf, rather than CMake.
39+
40+
??? List "Auxiliary TPLs Your System May Already Have Installed"
41+
42+
The following list of TPLs may be pre-installed on some systems, and are
43+
helpful, if not required, for building the netCDF libraries.
44+
However, in the author's experience they did need to be built on a fresh
45+
installation of Red Hat Enterprise Linux 8.
46+
47+
**Required**
48+
49+
- [LAPACK](https://www.netlib.org/lapack/) or [BLAS](https://www.netlib.org/blas/)
50+
- Alternatively, [OpenBLAS](http://www.openmathlib.org/OpenBLAS/),
51+
which is unlikely to be pre-installed.
52+
53+
**Likely Required**
54+
55+
- [Autoconf](https://www.gnu.org/software/autoconf/)
56+
- [Libtool](https://www.gnu.org/software/libtool/)
57+
58+
**Maybe Required**[^computers-amirite]
59+
60+
- [help2man](https://www.gnu.org/software/help2man/)
61+
62+
## Help & Hints
63+
64+
Building third-party libraries can be a pain.
65+
To help here are some reasonable example scripts to start from that should
66+
help getting the ***TPLs You Will Need to Build*** compiled and working.
67+
68+
??? Example "Basic Configuration Script Examples"
69+
70+
!!! Warning "Disclaimer"
71+
72+
We provide these scripts as demonstrative examples, only.
73+
They will almost certainly not build your libraries successfully if they
74+
are copy/paste-ed as they appear here.
75+
That being said, every hardware/software environment is different and
76+
will display different and sometimes strange behaviors.
77+
So, the best advice we can provide is to begin with something
78+
resembling what is below and respond to the compiler messages
79+
by modifying: environment variables, `PATH` or `[LD_]LIBRARY_PATH`
80+
entries, configuration arguments, and compiler arguments as required.
81+
As long as you do ***all of that*** perfectly, you should be just fine.
82+
:sunglasses:
83+
84+
### Assumptions
85+
86+
- Source code is downloaded to a directory we assign to the variable `${TPL_ROOT}`.
87+
- We build the libraries in a separate ***build*** and ***install***
88+
directory in the root directory of the source code.
89+
- E.g., `${TPL_ROOT}/<lib-name>/{build/,install/}`
90+
- The bash scripts below are placed in the `build/` directory.
91+
- Hence, note that when executing the configure script from the
92+
`build/` directory, the final `../configure [...]` command in the
93+
script (that calls autoconf) is running the `configure` command in
94+
the source directory for that TPL--e.g., `${TPL_ROOT}/<lib-name>/`
95+
- `libtool` is installed and on your `PATH`.
96+
- I.e., `which libtool` returns the location of the executable.
97+
98+
??? Abstract "`configure-hdf5.sh`"
99+
100+
```{.shell .copy}
101+
#!/bin/bash
102+
103+
# add MPI C/C++ compilers to the proper environment variables
104+
mpi_path="${TPL_ROOT}/openmpi/install/bin/"
105+
export CC="${mpi_path}/mpicc"
106+
export CXX="${mpi_path}/mpicxx"
107+
108+
# configure HDF5 build
109+
../configure \
110+
--prefix=${TPL_ROOT}/hdf5/install \
111+
--enable-shared \
112+
--enable-hl \
113+
--enable-parallel \
114+
--with-zlib=/usr/include,/usr/lib
115+
```
116+
117+
??? Abstract "`configure-netcdf-c.sh`"
118+
119+
```{.shell .copy}
120+
#!/bin/bash
121+
122+
# add MPI C/C++ to the proper environment variables
123+
mpi_path="${TPL_ROOT}/openmpi/install/bin"
124+
export CC="${mpi_path}/mpicc"
125+
export CXX="${mpi_path}/mpicxx"
126+
127+
# historically, netcdf can be a little picky to build, so it can help to
128+
# make ABSOLUTELY CERTAIN that the build system knows where HDF5 is
129+
# via the "sledgehammer" approach
130+
hdf5_path="${TPL_ROOT}/hdf5/install"
131+
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${hdf5_path}/lib"
132+
export LIBRARY_PATH=${LIBRARY_PATH}:${hdf5_path}/lib
133+
export PKG_CONFIG_PATH=${PKG_CONFIG_PATH}:${hdf5_path}/lib/pkgconfig
134+
export CPPFLAGS="-I${hdf5_path}/include"
135+
136+
../configure --prefix=${TPL_ROOT}/netcdf-c/install
137+
```
138+
139+
??? Abstract "`configure-netcdf-fortran.sh`"
140+
141+
```{.shell .copy}
142+
#! /bin/bash
143+
144+
# add MPI Fortran compiler to the proper environment variables
145+
mpi_path="${TPL_ROOT}/openmpi/install/bin"
146+
export FC="${mpi_path}/mpif90"
147+
148+
# local variables, to be used below
149+
export NetCDF_C_ROOT=${TPL_ROOT}/netcdf-c/install
150+
export NETCDF_C_LIBRARY=${NetCDF_C_ROOT}/lib
151+
export NETCDF_C_INCLUDE_DIR=${NetCDF_C_ROOT}/include
152+
153+
# add netcdf-C bits and pieces to the proper paths
154+
export LD_LIBRARY_PATH="${NETCDF_C_LIBRARY}:${LD_LIBRARY_PATH}"
155+
export LIBRARY_PATH="${NETCDF_C_LIBRARY}:${LIBRARY_PATH}"
156+
export PATH=${NetCDF_C_ROOT}/bin:${PATH}
157+
158+
# add netcdf-C include and library dir to the requisite compiler flags
159+
export CPPFLAGS="-I${NETCDF_C_INCLUDE_DIR}"
160+
export LDFLAGS="-L${NETCDF_C_LIBRARY}"
161+
162+
# while we're at it, tell it where to find hdf5, too
163+
export HDF5_ROOT=${TPL_ROOT}/hdf5/install
164+
165+
../configure --prefix=${TPL_ROOT}/netcdf-fortran/install
166+
```
167+
168+
??? Abstract "`configure-pnetcdf.sh`"
169+
170+
```{.shell .copy}
171+
#!/bin/bash
172+
173+
mpi_path="${TPL_ROOT}/openmpi/install/bin"
174+
export CC="${mpi_path}/mpicc"
175+
export CXX="${mpi_path}/mpicxx"
176+
export FC="${mpi_path}/mpif90"
177+
178+
../configure \
179+
--prefix=${TPL_ROOT}/pnetcdf/install \
180+
--enable-shared \
181+
MPICC=mpicc \
182+
MPICXX=mpicxx \
183+
MPIF77=mpifort \
184+
MPIF90=mpifort
185+
```
186+
187+
<!-- ======================================================================= -->
188+
189+
[^only-linux]: We only support build instructions for Linux, though some individuals have previously built EAMxx on MacOS with success.
190+
We would not recommend using EAMxx on Windows, other than via WSL.
191+
However, if you do end up building and running on MacOS or Windows,
192+
please let the developers know or submit a PR!
193+
<!-- markdownlint-disable-next-line MD053 -->
194+
[^computers-amirite]: Computers are weird. While it's by no means required from
195+
a functionality standpoint, a recently-configured RHEL8 system seemed to be
196+
convinced it needed help2man for some reason.

‎components/eamxx/docs/developer/dev_quickstart.md‎

Lines changed: 26 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ This means that, if you have an account on one of these machines, EAMxx tests
3939
can be run with no extra configuration required.
4040
However, many of the standalone EAMxx tests are able to be run on a
4141
reasonably-equipped laptop or personal workstation.[^but-mac]
42-
This second choice requires some extra configurations but the effort required
42+
This second choice requires some extra configurations[^TPLs-etc] but the effort required
4343
may be worth the potential speedup in development.
4444

4545
## Let's Get Started (Quickly)
@@ -50,11 +50,11 @@ may be worth the potential speedup in development.
5050
As such, we provide no guarantee that any of these are working code, and
5151
the author would be quite surprised if more than a couple of them run
5252
without edits.
53-
If you should take the time to debug these examples, feel free to submit a
53+
If you take the time to debug these examples, feel free to submit a
5454
PR with a working version, and we are sure some future developer will
5555
appreciate it!
5656

57-
### Running EAMxx via `test-all-eamxx`
57+
## Running EAMxx via `test-all-eamxx`
5858

5959
The quickest method for a new developer to get EAMxx up and running is to use
6060
the automated configure/build/test workflow provided by `scripts/test-all-eamxx`.
@@ -64,15 +64,27 @@ errors preventing successful running and testing.
6464

6565
Running via `test-all-eamxx` requires ***at minimum*** 1 to 2 command-line
6666
arguments to work correctly and likely more if your goals are more than modest.
67-
In the following sections, we present the configurations a new developer is most
68-
likely to require.
67+
In the following sections, we present the configurations a new developer is
68+
most likely to require.
6969

70-
#### Selected Useful Testing Options
70+
### Selected Useful Testing Options
7171

7272
We give an overview of common basic use cases here, and we provide full details
7373
on the various capabilities and options of `test-all-eamxx` in
7474
[EAMxx Automated Standalone Testing](dev_testing/test_all_eamxx.md).
7575

76+
!!! Warning "Note"
77+
78+
For all discussion of running standalone testing via `test-all-eamxx`,
79+
including the following, we assume the commands are run from the root
80+
directory of EAMxx. That is, if one begins in the root E3SM directory,
81+
assume that each code snippet begins with an implicit
82+
83+
``` {.shell .copy}
84+
$ eamxx_root="<some/file/path>/E3SM/components/eamxx"
85+
$ cd "${eamxx_root}"
86+
```
87+
7688
??? Example "Configure Only Without Build or Test"
7789

7890
**Useful for:** Mid-development testing when debugging compilation, and it
@@ -108,6 +120,7 @@ on the various capabilities and options of `test-all-eamxx` in
108120
# will run tests of the same configuration using N threads \
109121
--ctest-parallel-level <N> \
110122
```
123+
111124
??? Quote "Only Run Requested Test Cases"
112125

113126
As a first step, you are likely to only be concerned with testing your
@@ -248,7 +261,7 @@ on the various capabilities and options of `test-all-eamxx` in
248261
--make-parallel-level "${build_threads_gpu}"
249262
```
250263

251-
#### Testing on Supported Machines
264+
### Testing on Supported Machines
252265

253266
When testing on a supported E3SM machine, the only configuration required is to
254267
set the proper "machine ID" flag that indicates the machine name and potentially
@@ -286,7 +299,7 @@ and efficiently[^mach-file].
286299
$ ./scripts/test-all-eamxx --machine <machine-id>
287300
```
288301

289-
#### Testing Locally
302+
### Testing Locally
290303

291304
Running EAMxx on your local workstation, laptop, or less-trafficked cluster
292305
machine can potentially make your development quicker and less complicated.
@@ -296,8 +309,8 @@ setup involved in making your life easier.
296309
Testing on a local, or otherwise unknown, machine requires the `--local`
297310
(or `-l`) flag and correctly-configured files in the `~/.cime/` directory.
298311
For a full explanation of how to configure your own machine,
299-
see [Local Configuration Files](dev_testing/test_all_eamxx.md#local-
300-
configuration-files), but we will briefly summarize here.
312+
see the pages on [Local Configuration Files](dev_testing/test_all_eamxx.md#local-
313+
configuration-files) and [Third-party Libraries](TPLs.md), but we will briefly summarize here.
301314

302315
??? Example "Example `scream_mach_specs.py` Configuration File"
303316

@@ -330,8 +343,6 @@ configuration-files), but we will briefly summarize here.
330343
(mostly) more information is better than less.
331344

332345
```{.shell .copy title="${eamxx_root}/scripts/machine_specs.py:[L58-L88]"}
333-
from machines_specs import Machine
334-
335346
###############################################################################
336347
class Machine(object):
337348
###############################################################################
@@ -376,6 +387,8 @@ configuration-files), but we will briefly summarize here.
376387
concrete = True
377388
@classmethod
378389
def setup(cls):
390+
# NOTE: do not change this line because `test-all-eamxx -l`
391+
# specifically looks for a machine named "local".
379392
super().setup_base("local")
380393

381394
# NOTE: these are run by the shell (likely bash), so must have
@@ -417,18 +430,12 @@ configuration-files), but we will briefly summarize here.
417430
###############################################################################
418431
```
419432

420-
<!-- ## To-do
421-
422-
- Mention full-model, but don't go into detail
423-
- Discuss catch2 (add to comp. structures section)
424-
- `create_unit_test()`
425-
- Please use labels for tests -->
426-
427433
<!-- ======================================================================= -->
428434

429435
[^automagic-cmake]: In reality, significant effort is involved in designing a project to build "automagically," but the idea is to take that process out of the user's hands.
430436
[^supported-machines]: If you're feeling brave and want to read some python, you can find details about supported machines and their configurations in [machine_specs.py](https://github.com/E3SM-Project/E3SM/blob/master/components/eamxx/scripts/machines_specs.py) in the `scripts/` directory. Or, for general machine files for E3SM, you can take a look at `E3SM/cime_config/machines/`
431437
[^but-mac]: Mac computers with an Apple silicon CPU (e.g., M1, M2...) are not officially supported and present challenges related to handling floating-point exceptions. However, some have had success building on this architecture, and if you achieve a robust build configuration for Mac please reach out to the development team!
438+
[^TPLs-etc]: In particular, it is likely you will need to manually build/install the required [Third-party libraries](TPLs.md).
432439
[^smoke-test-def]: ***Smoke Test*** is a term used by software developers to describe a type of test that initially indicates whether something is working properly--as in, "flip the switch and see if anything starts smoking." :fingers_crossed:
433440
<!-- doesn't like the separation between footnote and text reference appears
434441
to be triggered by the admonition environment -->

‎components/eamxx/mkdocs.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,8 @@ nav:
3838
- 'Fields': 'developer/field.md'
3939
- 'Grids and Remappers': 'developer/grid.md'
4040
- 'Atmosphere Processes': 'developer/processes.md'
41-
- 'I/O': 'developer/io.md'
41+
- 'Input/Output': 'developer/io.md'
42+
- 'Third-party Libraries': 'developer/TPLs.md'
4243
- 'Technical Guide':
4344
- 'Overview': 'technical/index.md'
4445
- 'AeroCom cloud top': 'technical/aerocom_cldtop.md'

‎mkdocs.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,9 @@ markdown_extensions:
6363
generic: true
6464
- md_in_html
6565
- tables
66+
- pymdownx.emoji:
67+
emoji_index: !!python/name:material.extensions.emoji.twemoji
68+
emoji_generator: !!python/name:material.extensions.emoji.to_svg
6669

6770
plugins:
6871
- monorepo

0 commit comments

Comments
 (0)