347 conda packaging reboot

[ilaflott]

Description
This adds conda packaging workflows / configuration. follows up on #225

Will close #347

How Has This Been Tested?
See pipeline!

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published in downstream modules
• New check tests, if applicable, are included
• make distcheck passes

[ilaflott]

@mlee03 i'm largely trying to work out the conda-specifics here. the test failures... i'll be asking you et. al. for insight

[ilaflott]

speaking of test failures- a broken link smell emerges

2025-09-19T20:48:43.1265741Z WARNING: path /__w/FRE-NCtools/FRE-NCtools/tools/simple_hydrog/share/river_network_mrg_0.5deg_ad3nov_fill_coast_auto1_0.125.nc is a broken symlink - ignoring copy

[ilaflott]

sigh... for monday:

2025-09-19T21:02:41.2713625Z       File "/opt/software/linux-rocky9-zen2/gcc-11.4.1/miniconda3-24.7.1-akg5caxhoqv3oasili3pgukbyryceq74/lib/python3.12/site-packages/conda_build/create_test.py", line 86, in create_shell_files
2025-09-19T21:02:41.2714360Z         f.write(cmd)
2025-09-19T21:02:41.2714577Z     TypeError: write() argument must be str, not dict

[ilaflott]

I have quite a distribution of outcomes here...

pipeline, using fre-cli/miniconda docker image for linux/amd64

• build completes, with warnings
• tests, not even close:

# TOTAL: 83
# PASS:  22
# SKIP:  7
# XFAIL: 0
# FAIL:  37
# XPASS: 0
# ERROR: 17

the problems i can spot here are:

1. libstdc++.so.6: version GLIBCXX_3.4.30 not found , kills print_version tests, among quite a few others needed by libicuuc.so.75 (for some scatternc tests)
2. libstdc++.so.6: version CXXABI_1.3.15 not found , kills some scatternc tests i think, needed by libpnetcdf.so.7
3. some relative pathing gone awry with split_ncvars,

./init.sh: line 295: cd: <long>/conda-bld/fre-nctools_1758567516763/test_tmp/tests/../../src: No such file or directory 
split_ncvars: failed test: invalid path dir: ../../src

4. broken linking for certain testing files files (decomp, land)

local on my GFDL workstation

• build completes, with warnings
• ish, a lot more tests pass:

# TOTAL: 83
# PASS:  58
# SKIP:  7
# XFAIL: 0
# FAIL:  18
# XPASS: 0
# ERROR: 0

that was good enough i spun up a conda environment with fre-cli to test access to timavg. which timavg and which timavg.csh both work with the resultant conda-installed 2024.05.02 package, but run-time is a problem for fre-cli tests:

fre/app/generate_time_averages/frenctoolsTimeAverager.py:126: ValueError
--------------------------------------------------------------------------------------------------------------------------------------------------- Captured log call ---------------------------------------------------------------------------------------------------------------------------------------------------
WARNING  fre.app.generate_time_averages.frenctoolsTimeAverager:frenctoolsTimeAverager.py:49 Variable specification (var= alb_sfc) not currently supported for frenctols time averaging. ignoring!
ERROR    fre.app.generate_time_averages.frenctoolsTimeAverager:frenctoolsTimeAverager.py:124 stdout =

ERROR    fre.app.generate_time_averages.frenctoolsTimeAverager:frenctoolsTimeAverager.py:125 stderr =
 /home/Ian.Laflotte/conda/envs/fre_nctools_ians_local/libexec/fre-nctools/time_average: error while loading shared libraries: libnetcdf.so.22: cannot open shared object file: No such file or directory
timavg: time averaging executable failed with status 127
  command: /home/Ian.Laflotte/conda/envs/fre_nctools_ians_local/libexec/fre-nctools/time_average < timavg.nml.1Op9KumYOh
  timavg.nml.1Op9KumYOh:
    &input
    file_name_out = "/tmp/pytest-of-Ian.Laflotte/pytest-180/test_annual_av_from_monthly_ts0/av/180_288.conserve_order2/atmos_month/P1Y/P2Y/atmos_month.1980-1981.alb_sfc.nc",
    file_names = "merged_output.nc",
    use_end_time = .false.,
    verbose = .false.,
    add_cell_methods = .true.,
    skip_tavg_errors = .false.,
    suppress_warnings = .false.,
    frac_valid_data = 0.0,
    user_deflation = -1,
    user_shuffle = -1,
    command_line = "timavg -mb -o /tmp/pytest-of-Ian.Laflotte/pytest-180/test_annual_av_from_monthly_ts0/av/180_288.conserve_order2/atmos_month/P1Y/P2Y/atmos_month.1980-1981.alb_sfc.nc merged_output.nc",
    /

the good news here is that the tests predict this failure message in this context:

FAIL: list_ncvars/list_ncvars_ftn
=================================

list_ncvars: error while loading shared libraries: libnetcdff.so.7: cannot open shared object file: No such file or directory
list_ncvars_ftn: failed test: list_ncvars failed with no output
list_ncvars: error while loading shared libraries: libnetcdff.so.7: cannot open shared object file: No such file or directory
list_ncvars_ftn: failed test: list_ncvars failed printing 1 static variable
list_ncvars_ftn: failed test: list_ncvars should have printed 1 static vars, but printed 0 \n*****\n\n*****
FAIL list_ncvars/list_ncvars_ftn (exit status: 1)

there are similar error messages in the plevel routines, and the timavg tests do not pass

in this build, we also have the same issues as the github workflow does with split_ncvars' relative pathing, but, no GLIBCXX/CXXABI related errors- the library linking seemed to work as expected in the testiung context at the keyboard for my workstation. the print_version stuff worked fine too

cross-compile M-chip Mac w/ docker and hardware emulation

• build does not complete. i need to re-build the container from scratch and do so with the target platform in mind, with my own m-chip. so no-go.

on my wimpy older intel-chip macbook air, with docker

• build completes with warnings.
• [~] another one with almost-good enough results, this is the most reliable one so far. also taking the longest...

# TOTAL: 83
# PASS:  63
# SKIP:  7
# XFAIL: 0
# FAIL:  12
# XPASS: 0
# ERROR: 1

here i have the same relative-pathing problems as the other testing suite results, and a few GLIBCXX / CXXABI complaints. notably the timavg tests DO pass here, and so too do the list_ncvars and plevel tests... i should test this build if i can!

[ilaflott]

conclusions across the three contexts:

• the split_ncvars tests are going to fail and are irrelevant to the build context, and probably not the source of any conda build related issues
• CI/CD hates linked test data files, always
• every build is struggling with netcdf related errors or linking, but some more than others
• the biggest difference i can spot- the fre-cli container being built for amd64 is probably not entirely 1-to-1 with x86_64 for the purposes of conda packaging, and my local building on workstation and my intel mac may be leaning on system-installed things to close the gap in linked libraries