mppnccombine needs update to use the input format by default

[ceblanton]

Describe the bug
mppnccombine supports Netcdf-3 classic, Netcdf-3 64-bit offset, and Netcdf-4 input file and output file formats.

The FRE team believed that if you specify the output format (i.e. -64 or -n4), then the tool will use the format. Otherwise, it should use the input format.

Unfortunately, this is not how the tool works. How it works is: If you specify the output format (-64 or -n4), then the tool will use that format (Netcdf-3 64-bit offset and Netcdf-4 classic). But if the output format is not specified, then the default is Netcdf-3 classic.

To Reproduce
Find some test input files and convert them so both Netcdf-4 and Netcdf-3 64-bit offset are available.

Run mppncombine without specifying the output format, and observe the behavior.

In both cases, the tool chooses the never-desired Netcdf-3 classic format.

(use ncdump -k to see the file type)

Expected behavior
When the input files are either Netcdf-3 64-bit offset or regular Netcdf-3, and no output format is specified, then the output format should be Netcdf-3 64-bit offset.

When the input files are Netcdf-4 and no output format is specified, then the output format should be Netcdf-4.

System Environment
any

Additional context
Add any other context about the problem. If applicable, include where any files
that help describe, or reproduce the problem exist.

[ceblanton]

@HansVahlenkamp, as the original and most recent mppnccombine maintainer, could you fix this? Thank you!

[mlee03]

Hi @HansVahlenkamp, any update on this issue? 😁

[HansVahlenkamp]

Distracted with some other issues that needed attention. Will work on this next next week.

[mlee03]

@HansVahlenkamp, got it. I'll pester you next week then :)

[ceblanton]

@HansVahlenkamp , when you update mppnccombine, please come up with 3 clear test cases to show that the fix is good.

@underwoo volunteered to convert your documented test cases to unit tests, but he needs the 3 clear input files to do this.

Thank you! (and PS, I'll be pestering you too! we really need this update next week)

Test case 1: Netcdf-3 (not 64-bit) input: output should be Netcdf-3 64-bit

Test case 2: Netcdf-3 (64-bit) input: output should be Netcdf-3 64-bit

Test case 3: Netcdf-4 input: Netcdf-4 output

[mlee03]

@HansVahlenkamp, just checking, any updates?

[HansVahlenkamp]

@mlee03 I'm working on it this week and hope to have an updated version by next week...

Starting with version 2.2, there were significant structural changes done to the code by another author. The first step is reviewing those changes compared to the last version that I wrote. Also, my NetCDF test input files are quite old at this point and I could use some new ones (which are not too large) that represent all three of the test cases described by @ceblanton. Do you have any available?

[ceblanton]

@HansVahlenkamp I'm sure your old test files are OK but I'll dig up some new ones.

[ceblanton]

@HansVahlenkamp here are some distributed files on gaea, in the 3 formats:

gaea57:~%>ls /gpfs/f5/scratch/gfdl_f/Chris.Blanton/test-files/*                                                                                                                                                                                   1079
/gpfs/f5/scratch/gfdl_f/Chris.Blanton/test-files/nc3:
out.nc	vegn2.res.tile6.nc.0000  vegn2.res.tile6.nc.0001  vegn2.res.tile6.nc.0002  vegn2.res.tile6.nc.0003  vegn2.res.tile6.nc.0004

/gpfs/f5/scratch/gfdl_f/Chris.Blanton/test-files/nc3-64:
out.nc	vegn2.res.tile6.nc.0000  vegn2.res.tile6.nc.0001  vegn2.res.tile6.nc.0002  vegn2.res.tile6.nc.0003  vegn2.res.tile6.nc.0004

/gpfs/f5/scratch/gfdl_f/Chris.Blanton/test-files/nc4:
out.nc	vegn2.res.tile6.nc.0000  vegn2.res.tile6.nc.0001  vegn2.res.tile6.nc.0002  vegn2.res.tile6.nc.0003  vegn2.res.tile6.nc.0004

[mlee03]

Hi @HansVahlenkamp, any chance the PR can be made in the next couple of days?

[HansVahlenkamp]

@mlee03 Hopefully done in the next couple of days.

[mlee03]

@HansVahlenkamp, I'm back! Any updates? Is it OK if you open up a draft PR as a placeholder?

[HansVahlenkamp]

@mlee03 It turns out to be a bit more complicated than originally thought. The code creates the output file with a particular format before any of the input files are read. So, some code restructuring is needed to make this flexible based upon the format of the first input file. I'm currently testing some ideas for this restructuring. I was also investigating some other cleanup of the code, but that probably should wait until another time to minimize any further delays in resolving the main issue.