prepare release of 1.14.1

Author: wkliao

DEVELOPER_NOTES.md

@@ -7,7 +7,11 @@
     codes.  PnetCDF uses the same constants and error codes as NetCDF. See
     "Conform with netCDF" below.
 
- 2. Set the release version
+ 2. create a new branch, for example named "prepare-1.14.1"
+    git checkout -B prepare-1.14.1
+    git add the bullet points 3 and 4 below to the new branch
+
+ 3. Set the release version
     In file `configure.ac`, variable `$PNETCDF_VERSION` indicates the version.
     It is automatically generated from the 2nd argument of `AC_INIT` call.
     Revise that string to the right release version. For example,
@@ -17,15 +21,15 @@
     ```
     For setting shared library version (ABI versioning), see below.
 
- 3. Update file `RELEASE_NOTES`
+ 4. Update file `RELEASE_NOTES`
     Copy the contents of file `sneak_peek.md` to the top of file `RELEASE_NOTES`.
     Clear up file `sneak_peek.md` (reset all items to none).
     + 1.9.0 and later - the release version and date in `RELEASE_NOTES` will
       be automatically set when running command `make dist`.
     + 1.8.1 and priors - add the release version and date at the top
       of file `RELEASE_NOTES`.
 
- 4. Update the release date (1.8.1 and priors)
+ 5. Update the release date (1.8.1 and priors only)
     For more info, see below section "Setting PnetCDF software release date".
     Run command `svn commit` to get the svn property \$LastChangedDate\$
     updated in file `configure.in`. That string will be used as the official
@@ -34,14 +38,17 @@
     svn commit -m "set release date of version 1.6.1 to today"
     ```
 
- 5. Commit all changes to repo servers
+ 6. Commit all changes to repo servers
     * 1.8.1 and priors -- run `svn commit` to upload changes to SVN server.
-    * 1.9.0 and after -- run `git push origin master` to upload changes to
-      github.com.
+    * 1.9.0 and after --
+        + `git commit -m "prepare release of 1.14.1"
+        + `git push origin prepare-1.14.1` to upload changes to github.com.
+        + Create a pull request at github.com
+        + When all github actions passed, merge the PR.
 
- 6. Generate release tar ball (use it for testing)
+ 7. Generate release tar ball (use it for testing)
     * Generate a new `configure` file
-      + Run command `autoreconf` to generate file `configure` to be included
+      + Run command `autoreconf -i` to generate file `configure` to be included
         in the release (but ignored by the SVN/Git repo).
       + Must use the bug-fixed autotools of version 2.69 or higher to run
         `autoreconf`. See [README.Fujitsu](doc/README.Fujitsu) for the bug in
@@ -57,7 +64,7 @@
         `make dist`. This setting will be done automatically, unlike step 4
         above that manually update the release date.
 
- 7. Test the new release tar ball (not source codes from the repo)
+ 8. Test the new release tar ball (not source codes from the repo)
     * build under the same directory as source
       - run `configure` (with command-line options: `--enable-strict`,
         `--enable-coverage`, `--disable-cxx`, `--disable-fortran` and their
@@ -125,7 +132,7 @@
       run instructions. Remember to add new test/example programs added in the
       new release.
 
- 8. Create a checkpoint
+ 9. Create a checkpoint
     * For 1.9.0 and priors - Create a new SVN tag on svn repo, by running
       command below to duplicate the current trunk to a new tag:
       ```
@@ -139,7 +146,7 @@
       git tag -a checkpoint.1.11.0 -m "Checkpoint right before 1.11.0 release"
       git push origin checkpoint.1.11.0
       ```
- 9. Generate SHA1 checksums
+ 10. Generate SHA1 checksums
     * Run command:
       ```
       openssl sha1 pnetcdf-1.11.0.tar.gz`
@@ -148,7 +155,7 @@
       ```
       SHA1(pnetcdf-1.11.0.tar.gz)= 495d42f0a41abbd09d276262dce0f7c1c535968a
       ```
-10. Update PnetCDF Web Page
+11. Update PnetCDF Web Page
     * https://github.com/Parallel-NetCDF/Parallel-NetCDF.github.io
     * Create a new file of release note Parallel-NetCDF.github.io/Release_notes/1.11.0.md.
     * Add a news item in index.html to announce the new release version.
@@ -208,6 +215,23 @@
          the old version at runtime: set revision to 0, bump current and age.
       3. Programs may need to be changed, recompiled, and relinked in order to
          use the new version. Bump current, set revision and age to 0.
+
+   libtool Chapter 7.2 Libtool’s versioning system
+      So, libtool library versions are described by three integers:
+
+      current
+          The most recent interface number that this library implements.
+
+      revision
+          The implementation number of the current interface.
+
+      age
+          The difference between the newest and oldest interfaces that this
+          library implements. In other words, the library implements all the
+          interface numbers in the range from number current - age to current.
+
+      If two libraries have identical current and age numbers, then the dynamic
+      linker chooses the library with the greater revision number.
   ```
 ---
 ### Note on adding new MPI compiler candidates
@@ -319,6 +343,9 @@
 
 ---
 ### config.guess, config.sub, install-sh in directory scripts
+There 3 files are automatically added when running command "autoreconf -i"
+Below is just FYI.
+
 Copy config.guess, config.sub, and install-sh from GNU libtool.
 http://www.gnu.org/software/libtool/
 

README.md

@@ -60,6 +60,7 @@ Northwestern University and Argonne National Laboratory.
 * [Utility Programs](./src/utils#readme)
 * [NetCDF4 vs. PnetCDF](./doc/netcdf4_vs_pnetcdf.md)
 * PnetCDF [blocking vs. non-blocking APIs](./doc/blocking_vs_nonblocking.md)
+* [CDL header API references](./doc/cdl_api_guide.md)
 
 ### Mailing List
 * parallel-netcdf@mcs.anl.gov

RELEASE_NOTES

@@ -6,6 +6,113 @@ PnetCDF Release Notes
 Version _PNETCDF_VERSION_ (_PNETCDF_RELEASE_DATE_)
 -------------------------------------
 
+* New optimization
+  + When file header extent size grows, moving the data section to a higher
+    file offset has changed to be done in chunks of 16 MB per process.
+    See [PR #174](https://github.com/Parallel-NetCDF/PnetCDF/pull/174),
+
+* Configure options
+  + For PnetCDF developers, the requirement for libtool version has been
+    changed to 2.5.4, due to an issue on Mac OS when using OpenMPI. See
+    [Issue #155](https://github.com/Parallel-NetCDF/PnetCDF/issues/155),
+    [Issue #163](https://github.com/Parallel-NetCDF/PnetCDF/issues/163),
+    and [PR #164](https://github.com/Parallel-NetCDF/PnetCDF/pull/164).
+
+* New APIs
+  + A set of APIs that read the header of a CDL file header and allow users to
+    query the metadata defined in the CDL file. These APIs can be useful for
+    creating a program (such as an I/O benchmark) based on an output netCDF
+    file from another application (a CDL file can be generated by running the
+    utility `ncmpidump/ncdump`). See
+    [PR #177](https://github.com/Parallel-NetCDF/PnetCDF/pull/177).
+    * `cdl_hdr_open()` opens and parses the CDL file's header
+    * `cdl_hdr_inq_format()` returns file format version
+    * `cdl_hdr_inq_ndims()` returns number of dimensions defined in CDL file
+    * `cdl_hdr_inq_dim()` returns metadata of a dimension
+    * `cdl_hdr_inq_nvars()` returns number of variables
+    * `cdl_hdr_inq_var()` returns metadata of a variable defined in CDL file
+    * `cdl_hdr_inq_nattrs()` returns number of attributes of a given variable
+    * `cdl_hdr_inq_attr()` returns metadata of an attribute
+    * `cdl_hdr_close()` closes the CDL file
+  + The CDL Header C Reference Manual is available in
+    [cdl_api_guide.md](https://github.com/Parallel-NetCDF/PnetCDF/blob/master/doc/cdl_api_guide.md).
+    See [PR #184](https://github.com/Parallel-NetCDF/PnetCDF/pull/184).
+
+* Bug fixes
+  + Fix a GCC 15 compilation error. Thanks to Adrian Reber.
+    See [PR #179](https://github.com/Parallel-NetCDF/PnetCDF/pull/179).
+  + Fix errors when building an RPM. Thanks to Adrian Reber.
+    See [PR #178](https://github.com/Parallel-NetCDF/PnetCDF/pull/178).
+  + Fix setting of user hint nc_ibuf_size.
+    See [PR #161](https://github.com/Parallel-NetCDF/PnetCDF/pull/161).
+
+* New example programs
+  + examples/C/create_from_cdl.c shows how to call the new CDL header APIs to
+    create a netCDF file with the same metadata.
+
+* New programs for I/O benchmarks
+  + WRF-IO contains an extraction of the I/O kernel of WRF (Weather Research
+    and Forecast Model, a weather prediction computer simulation program
+    developed at NCAR) that can be used to evaluate the file I/O performance
+    of WRF. It's data partitioning pattern is a 2D block-block checkerboard
+    pattern, along the longitude and latitude.
+    See [PR #165](https://github.com/Parallel-NetCDF/PnetCDF/pull/165)
+    and [PR #181](https://github.com/Parallel-NetCDF/PnetCDF/pull/181).
+
+* New test program
+  + test/cdf/tst_cdl_hdr_parser.c tests the new CDL header APIs.
+  + test/testcases/tst_grow_header.c tests header extent growth by re-entering
+    the define mode multiple times and add more fix-sized and record variables.
+
+* Clarifications about of PnetCDF hints
+  + There are three ways in PnetCDF for user to set hints to align the starting
+    file offset for the data section (header extent) and record variable
+    section.
+    1. through a call to API `nc_header_align_size` by setting arguments of
+       `h_minfree`, `v_align`, `v_minfree`, and `r_align`.
+    2. through an MPI info object passed to calls of `ncmpi_create()` and
+       `ncmpi_open()`. Hints are `nc_header_align_size`, `nc_var_align_size`,
+       and `nc_record_align_size`.
+    3. through a run-time environment variable `PNETCDF_HINTS`. Hints are
+       `nc_header_align_size`, `nc_var_align_size`, and `nc_record_align_size`.
+  + As the same hints may be set by one or more of the above methods, PnetCDF
+    implements the following hint precedence.
+    * `PNETCDF_HINTS` > `ncmpi__enddef()` > `MPI info`.
+    * 1st priority: hints set in the environment variable `PNETCDF_HINTS`, e.g.
+      `PNETCDF_HINTS="nc_var_align_size=1048576"`. Making this the first
+      priority is because it allows to run the same application executable
+      without source code modification using different alignment settings
+      through a run-time environment variable.
+    * 2nd priority: hints set in the MPI info object passed to calls of
+      `ncmpi_create()` and `ncmpi_open()`, e.g.
+      `MPI_Info_set("nc_var_align_size", "1048576");`. The reasoning is when a
+      3rd-party library built on top of PnetCDF implements its codes using
+      'ncmpi__enddef'. An application that uses such 3rd-party library can pass
+      an MPI info object to it, which further passes the info to PnetCDF. This
+      precedence allows that application to exercise different hints without
+      changing the 3rd-party library's source codes.
+    * 3rd priority: hints used in the arguments of `ncmpi__enddef()`, e.g.
+      `ncmpi__enddef(..., v_align=1048576,...)`.
+  + PnetCDF I/O hint `nc_header_align_size` is essentially the same as hint
+    `nc_var_align_size`, but its name appears to be closer to the hint's
+    intent, i.e. to reserve some space for the header growth in the future when
+    new data objects are added. Please note when both hints are set, only hint
+    `nc_var_align_size` will take effect and `nc_header_align_size` ignored.
+  + When there is no fix-sized variable (i.e. non-record variable) defined,
+    argument `v_minfree` passed to `ncmpi__enddef()` is ignored. In this
+    case, users should set `h_minfree`, if an extra header space is desired.
+  + When there is no fix-sized variables defined and none of hints
+    `nc_header_align_size`, `nc_var_align_size`, or argument `v_align` is set,
+    `nc_record_align_size` or `r_align`, if set, will be used to align the
+    header extent.
+  + For the above update to the hint precedence, see
+    PnetCDF See [PR #173](https://github.com/Parallel-NetCDF/PnetCDF/pull/173).
+
+
+-------------------------------------
+Version 1.14.0 (November 11, 2024)
+-------------------------------------
+
 * New features
   + Intra-node aggregation for write requests -- When the number of MPI
     processes allocated to a compute node is large, this feature can

benchmarks/README.md

@@ -12,7 +12,7 @@ using various I/O access patterns.
 ---
 
 ### WRF-IO
-   + WRF (Wether Research and Forecast Model) is a weather prediction computer
+   + WRF (Weather Research and Forecast Model) is a weather prediction computer
      simulation program, https://github.com/wrf-model/WRF, developed at NCAR.
    + This benchmark is an extraction of the I/O kernel of WRF and can be used
      to evaluate the file write performance of WRF.

benchmarks/WRF-IO/wrf_io.c

@@ -6,7 +6,7 @@
  * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
 
 /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
- * This program evaluates the file write and read performance of WRF (Wether
+ * This program evaluates the file write and read performance of WRF (Weather
  * Research and Forecast Model, https://github.com/wrf-model/WRF) developed at
  * NCAR. It's data partitioning pattern is a 2D block-block checkerboard
  * pattern, along the longitude and latitude. This benchmark program reads a

configure.ac

@@ -15,7 +15,10 @@ dnl AC_REVISION([$Revision$])dnl
 dnl autoconf v2.70 and later is required. See https://github.com/Parallel-NetCDF/PnetCDF/issues/94
 dnl autoconf v2.70 was released in 2021-01-28
 AC_PREREQ([2.70])
-AC_INIT([PnetCDF],[1.14.0],[parallel-netcdf@mcs.anl.gov],[pnetcdf],[https://parallel-netcdf.github.io])
+AC_INIT([PnetCDF], [1.14.1],
+        [parallel-netcdf@mcs.anl.gov],
+        [pnetcdf],
+        [https://parallel-netcdf.github.io])
 
 dnl config.h.in will be created by autoreconf (autoheader)
 dnl call it right after AC_INIT, as suggested by autoconf
@@ -2643,7 +2646,7 @@ dnl Update the version information only immediately before a public release.
 dnl PnetCDF starts with 1:0:0 (shared library is first supported in 1.9.0)
 dnl because some package distributors, such as Debian, may have already built
 dnl PnetCDF with shared libraries.
-ABIVERSION="6:0:0"
+ABIVERSION="7:0:0"
 AC_SUBST(ABIVERSION)
 if test "$enable_versioning" = "yes" ; then
    ABIVERSIONFLAGS="-version-info \$(ABIVERSION)"