Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
93f9cda
#224 Added basic conda build recipe and build script.
May 5, 2023
623c055
Merge remote-tracking branch 'chris_fork/224.conda' into 347-conda-pa…
ilaflott Jan 30, 2025
5f61648
add build_conda
ilaflott Jan 30, 2025
5df2f0b
disable workflows i am not touching- add jinja to meta.yaml to grab g…
ilaflott Jan 30, 2025
7ea36da
use the fre-cli container with miniconda installed instead
ilaflott Jan 30, 2025
9c80fb8
remove git tag jinja2 reference... apparently that needs a setup.py a…
ilaflott Jan 30, 2025
9719b46
comment out matrix running for now- add appropriate conda-forge packa…
ilaflott Jan 30, 2025
f59330d
update build reqs- adjust build target for hdf5 library, add openmp
ilaflott Jan 30, 2025
481a1bd
add tcsh
ilaflott Jan 30, 2025
6298e24
Create split_ncvars.pl.1
ilaflott Feb 10, 2025
dde7244
Update Makefile.am
ilaflott Feb 10, 2025
5c8c20b
Update build.sh
ilaflott Feb 10, 2025
3415a99
guard against lack of split_ncvars man page in conda package, at auto…
ilaflott Feb 10, 2025
c054d77
remove non-existant option in configure call. re-autoconfigure in tes…
ilaflott Feb 10, 2025
de7c204
Merge branch 'main' into 347-conda-packaging-reboot
ilaflott Sep 17, 2025
6b462bc
unparallelize the build for simplicity, line up the conda build envir…
ilaflott Sep 17, 2025
5011e5b
remove defaults channel just in case, and realllllly get the conda bu…
ilaflott Sep 17, 2025
f35a178
forget netcdf4 for a minute
ilaflott Sep 17, 2025
92fa3e1
choose the fre-nctools container instead of the fre-cli one. restore …
ilaflott Sep 17, 2025
bd9dfc3
fine, disable all the workflows other than the conda building one. i …
ilaflott Sep 17, 2025
be1a281
ok, back to fre-cli container...
ilaflott Sep 17, 2025
6e44dd0
back to normal conda-verify and conda-build
ilaflott Sep 17, 2025
e1eb0ea
this should fix the docbook oriented complaint
ilaflott Sep 17, 2025
64dcdb9
simplify metayaml recipe a touch
ilaflott Sep 17, 2025
b45b5fb
further simplify env. also specifically ask for python 3
ilaflott Sep 17, 2025
98c162d
closer...
ilaflott Sep 17, 2025
a934edb
try without autoreconf
ilaflott Sep 17, 2025
34ce960
closer... try testing as part of the build. not conda canon, but oh wlel
ilaflott Sep 17, 2025
aedaf5d
so close
ilaflott Sep 17, 2025
f5aeefb
cmooooon!
ilaflott Sep 17, 2025
21e1df6
CMOOON
ilaflott Sep 17, 2025
aba7f8a
Initial plan
Copilot Sep 17, 2025
da32f41
Fix conda package tests to avoid make check in test phase
Copilot Sep 17, 2025
a23578e
Add comprehensive conda package tests for key FRE-NCtools programs
Copilot Sep 17, 2025
2c78f41
Update build_conda workflow trigger to push
ilaflott Sep 18, 2025
0ff892d
Uncomment main.yml workflow and add functional tests for timavg and c…
Copilot Sep 18, 2025
4b3a11f
Simplify conda tests to use single-line commands instead of multiline…
Copilot Sep 18, 2025
f8b7514
Re-comment main.yml workflow and fix meta.yaml tests to use make chec…
Copilot Sep 18, 2025
02e53c5
Fix conda test autoreconf issue by using simple functional tests with…
Copilot Sep 18, 2025
0d0b028
Revert "Fix conda test autoreconf issue by using simple functional te…
ilaflott Sep 18, 2025
968c6b5
Merge pull request #376 from NOAA-GFDL/copilot/fix-34729303-cdef-45a4…
ilaflott Sep 18, 2025
cf93f7e
clearly print out some helpful info
ilaflott Sep 18, 2025
937047a
Include source_files for testing configuration
ilaflott Sep 18, 2025
9ff9653
Add source files for testing in meta.yaml
ilaflott Sep 18, 2025
5e840e3
Refactor test environment setup commands
ilaflott Sep 18, 2025
3ad0c66
Add source files pattern to test configuration
ilaflott Sep 18, 2025
4723b0e
Update test source files in meta.yaml
ilaflott Sep 18, 2025
4de8a9e
Add man directory to test source files
ilaflott Sep 18, 2025
bf77ac3
Refactor requirements in meta.yaml
ilaflott Sep 18, 2025
2430409
Update meta.yaml
ilaflott Sep 18, 2025
208cdae
Update meta.yaml with additional conda requirements
ilaflott Sep 18, 2025
087afde
local build on workstation quite similar to build in pipeline now! cl…
ilaflott Sep 19, 2025
c43952d
undo older edits to files, unsure if necessary
ilaflott Sep 19, 2025
977a570
delete a file i created to fix idk what. update env yaml, maybe i wil…
ilaflott Sep 19, 2025
b94005d
missed a bit
ilaflott Sep 19, 2025
3e7f83b
lets not get too complicated too quick
ilaflott Sep 19, 2025
de3dc9a
syntax for flag and lets try the --test flag and see what happens
ilaflott Sep 19, 2025
65b092f
forget splitting the build and test step for a minute... sigh
ilaflott Sep 19, 2025
90b453f
syntax error?
ilaflott Sep 19, 2025
46736fa
comment out the test section for a second, lets see what happens
ilaflott Sep 19, 2025
3e20bca
bring back testing step and exit guard the make check. copy log file …
ilaflott Sep 19, 2025
124f81a
fix tarball path, uncomment test-suite.log upload that we are gonna t…
ilaflott Sep 19, 2025
1324832
friday YOLO: try the build directory option.
ilaflott Sep 19, 2025
e41eaa3
Enhance build script output messages
ilaflott Sep 22, 2025
64387f7
Update conda build command with debug option
ilaflott Sep 22, 2025
7495052
Revert "Update conda build command with debug option"
ilaflott Sep 22, 2025
3f56785
Revert "Enhance build script output messages"
ilaflott Sep 22, 2025
fea46cc
Revert "friday YOLO: try the build directory option."
ilaflott Sep 22, 2025
acf7ed4
Revert "fix tarball path, uncomment test-suite.log upload that we are…
ilaflott Sep 22, 2025
1e7d399
Revert "bring back testing step and exit guard the make check. copy l…
ilaflott Sep 22, 2025
118bfa9
Revert "comment out the test section for a second, lets see what happ…
ilaflott Sep 22, 2025
85871cb
re-introduce potentially offensive line into metayaml, and also a ha…
ilaflott Sep 22, 2025
11ec177
try reintroducing the correct path for the tarball in the artifact up…
ilaflott Sep 22, 2025
3bd00f6
now turn on the test-suite log uploading
ilaflott Sep 22, 2025
209f6fe
emacs' whitespace-cleanup on OSX is introducing weird whitespace char…
ilaflott Sep 22, 2025
5469a5d
re-do the test-suite log upload bit
ilaflott Sep 22, 2025
49fc134
many errors in the test-suite.log seem related to GLIBCXX and CXXABI …
ilaflott Sep 22, 2025
96439e4
ugh whitespace killing me
ilaflott Sep 22, 2025
2a86e69
try pinning the gcc compiler. try setting the LD_LIBRARY_PATH to PREF…
ilaflott Sep 22, 2025
3b58051
shot in the dark- remove openmp and mpi packages
ilaflott Sep 22, 2025
e8ed381
put mpich (implementation of mpi on conda-forge) back in reqs
ilaflott Sep 22, 2025
73cb808
libhdf5 add in...
ilaflott Sep 22, 2025
06d73de
correct the hdf5 req
ilaflott Sep 22, 2025
c572fa5
correct the hdf5 req... again
ilaflott Sep 22, 2025
9ac299b
fix gcc version to whats in the container
ilaflott Sep 22, 2025
8f10580
remove gcc req, just use whatever is in the container.. re-add back i…
ilaflott Sep 22, 2025
933aa44
i think the ld_library_path thing is not gonna work for us...
ilaflott Sep 22, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .github/workflows/build_conda.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: build_conda

on: [push]

# pull_request:
# branches:
# - main

jobs:
build:
runs-on: ubuntu-latest

container:
image: ghcr.io/noaa-gfdl/fre-cli:miniconda24_gcc14_v2
# image: ghcr.io/noaa-gfdl/fre-cli:miniconda24.7.1_gcc14.2.0
# image: ghcr.io/noaa-gfdl/fre-nctools-ci-rocky-gnu:14.2.0_v2

steps:
- name: Checkout Files
uses: actions/checkout@v4
with:
submodules: 'recursive'

- name: Run Conda to Build
continue-on-error: true
run: |
# append the reqd channels
conda config --append channels conda-forge
conda config --append channels noaa-gfdl

# remove any default channels
conda config --remove channels defaults

# just in case
conda config --show channels

# install conda-build and conda-verify
conda install conda-build conda-verify

# conda build
mkdir -p /app/fre-nctools-tarball
conda build . --package-format tar.bz2 --output-folder /app/fre-nctools-tarball || echo "conda build failed, guarding. artifact upload will fail instead."
#conda build . --build-only --no-anaconda-upload --package-format tar.bz2 --output-folder /app/fre-nctools-tarball || echo "conda build failed, guarding. artifact upload will fail instead."
#conda build --test $(ls /app/fre-nctools-tarball/noarch/fre-nctools-*.tar.bz2)

- name: Upload fre-nctools tarball
uses: actions/upload-artifact@v4
with:
name: fre-nctools-tarball
path: /app/fre-nctools-tarball/linux-64/fre-nctools-*.tar.bz2
if-no-files-found: error

# highly desired...
- name: Upload test-suite.log
uses: actions/upload-artifact@v4
with:
name: fre-nctools-test-suite-log
path: /app/fre-nctools-tarball/test-suite.log
if-no-files-found: error
85 changes: 43 additions & 42 deletions .github/workflows/distcheck.yml
Original file line number Diff line number Diff line change
@@ -1,42 +1,43 @@
# Github actions CI file
# Ryan Mulhall 2020
# CI testing for the FRE-NCtools repo, builds and runs unit tests
# image dockerfile is maintained here:
# https://gitlab.gfdl.noaa.gov/fre/hpc-me
name: FRE-NCtools CI
on:
workflow_run:
workflows: ["FRE-NCtools Check Expensive"]
types:
- completed
jobs:
CI:
runs-on: ubuntu-latest
strategy:
matrix:
with_mpi: ['','--with-mpi']
enable_quad_precision: ['', '--enable-quad-precision']
container:
image: ghcr.io/noaa-gfdl/fre-nctools-ci-rocky-gnu:14.2.0_v2
env:
MPI: ${{ matrix.with_mpi }}
QUAD_P: ${{ matrix.enable_quad_precision }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive
- name: Configure
run: |
mkdir build && cd build
autoreconf -i ../configure.ac
../configure $MPI $QUAD_P || cat config.log
- name: Build distribution with check
run: make DISTCHECK_CONFIGURE_FLAGS="$MPI $QUAD_P" -C build -j
- name: Save log file on failure
uses: actions/upload-artifact@v4.4.0
if: failure()
with:
name: test-suites
path: |
build/tests/test-suite.log
## Github actions CI file
## Ryan Mulhall 2020
## CI testing for the FRE-NCtools repo, builds and runs unit tests
## image dockerfile is maintained here:
## https://gitlab.gfdl.noaa.gov/fre/hpc-me
#name: FRE-NCtools CI
#on:
# workflow_run:
# workflows: ["FRE-NCtools Check Expensive"]
# types:
# - completed
#jobs:
# CI:
# runs-on: ubuntu-latest
# strategy:
# matrix:
# with_mpi: ['','--with-mpi']
# enable_quad_precision: ['', '--enable-quad-precision']
# container:
# image: ghcr.io/noaa-gfdl/fre-nctools-ci-rocky-gnu:14.2.0_v2
# env:
# MPI: ${{ matrix.with_mpi }}
# QUAD_P: ${{ matrix.enable_quad_precision }}
# steps:
# - name: Checkout
# uses: actions/checkout@v4
# with:
# submodules: recursive
# - name: Configure
# run: |
# mkdir build && cd build
# autoreconf -i ../configure.ac
# ../configure $MPI $QUAD_P || cat config.log
# - name: Build distribution with check
# run: make DISTCHECK_CONFIGURE_FLAGS="$MPI $QUAD_P" -C build -j
# - name: Save log file on failure
# uses: actions/upload-artifact@v4.4.0
# if: failure()
# with:
# name: test-suites
# path: |
# build/tests/test-suite.log
#
84 changes: 42 additions & 42 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -1,42 +1,42 @@
# Github actions CI file
# Ryan Mulhall 2020
# CI testing for the FRE-NCtools repo, builds and runs unit tests
# image dockerfile is maintained here:
# https://gitlab.gfdl.noaa.gov/fre/hpc-me
name: FRE-NCtools CI
on: [push, pull_request]
jobs:
CI:
runs-on: ubuntu-latest
strategy:
matrix:
with_mpi: ['','--with-mpi']
enable_quad_precision: ['', '--enable-quad-precision']
container:
image: ghcr.io/noaa-gfdl/fre-nctools-ci-rocky-gnu:14.2.0_v2
env:
MPI: ${{ matrix.with_mpi }}
QUAD_P: ${{ matrix.enable_quad_precision }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive
- name: Configure
run: |
mkdir build && cd build
autoreconf -i ../configure.ac
../configure $MPI $QUAD_P || cat config.log
- name: Build
# Due to how some compilers handle the Fortran module files, we
# need to run the build twice
run: make -C build -j || make -C build -j
- name: Run most tests (skip the slow ones)
run: make -C build -j check
- name: Save log file on failure
uses: actions/upload-artifact@v4.4.0
if: failure()
with:
name: test-suites
path: |
build/tests/test-suite.log
## Github actions CI file
## Ryan Mulhall 2020
## CI testing for the FRE-NCtools repo, builds and runs unit tests
## image dockerfile is maintained here:
## https://gitlab.gfdl.noaa.gov/fre/hpc-me
#name: FRE-NCtools CI
#on: [push, pull_request]
#jobs:
# CI:
# runs-on: ubuntu-latest
# strategy:
# matrix:
# with_mpi: ['','--with-mpi']
# enable_quad_precision: ['', '--enable-quad-precision']
# container:
# image: ghcr.io/noaa-gfdl/fre-nctools-ci-rocky-gnu:14.2.0_v2
# env:
# MPI: ${{ matrix.with_mpi }}
# QUAD_P: ${{ matrix.enable_quad_precision }}
# steps:
# - name: Checkout
# uses: actions/checkout@v4
# with:
# submodules: recursive
# - name: Configure
# run: |
# mkdir build && cd build
# autoreconf -i ../configure.ac
# ../configure $MPI $QUAD_P || cat config.log
# - name: Build
# # Due to how some compilers handle the Fortran module files, we
# # need to run the build twice
# run: make -C build -j || make -C build -j
# - name: Run most tests (skip the slow ones)
# run: make -C build -j check
# - name: Save log file on failure
# uses: actions/upload-artifact@v4.4.0
# if: failure()
# with:
# name: test-suites
# path: |
# build/tests/test-suite.log
89 changes: 45 additions & 44 deletions .github/workflows/main_expensive.yml
Original file line number Diff line number Diff line change
@@ -1,44 +1,45 @@
# Github actions CI file
# Ryan Mulhall 2020
# CI testing for the FRE-NCtools repo, builds and runs unit tests
# image dockerfile is maintained here:
# https://gitlab.gfdl.noaa.gov/fre/hpc-me
name: FRE-NCtools Check Expensive
on:
workflow_run:
workflows: ["FRE-NCtools CI"]
types:
- completed
jobs:
CI:
runs-on: ubuntu-latest
strategy:
matrix:
with_mpi: ['','--with-mpi']
enable_quad_precision: ['', '--enable-quad-precision']
container:
image: ghcr.io/noaa-gfdl/fre-nctools-ci-rocky-gnu:14.2.0_v2
env:
MPI: ${{ matrix.with_mpi }}
QUAD_P: ${{ matrix.enable_quad_precision }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive
- name: Configure
run: |
mkdir build && cd build
autoreconf -i ../configure.ac
../configure $MPI $QUAD_P || cat config.log
- name: Build
run: make -C build -j
- name: Run most tests (skip the slow ones)
run: make -C build check-very-expensive
- name: Save log file on failure
uses: actions/upload-artifact@v4.4.0
if: failure()
with:
name: test-suites
path: |
build/tests/test-suite.log
## Github actions CI file
## Ryan Mulhall 2020
## CI testing for the FRE-NCtools repo, builds and runs unit tests
## image dockerfile is maintained here:
## https://gitlab.gfdl.noaa.gov/fre/hpc-me
#name: FRE-NCtools Check Expensive
#on:
# workflow_run:
# workflows: ["FRE-NCtools CI"]
# types:
# - completed
#jobs:
# CI:
# runs-on: ubuntu-latest
# strategy:
# matrix:
# with_mpi: ['','--with-mpi']
# enable_quad_precision: ['', '--enable-quad-precision']
# container:
# image: ghcr.io/noaa-gfdl/fre-nctools-ci-rocky-gnu:14.2.0_v2
# env:
# MPI: ${{ matrix.with_mpi }}
# QUAD_P: ${{ matrix.enable_quad_precision }}
# steps:
# - name: Checkout
# uses: actions/checkout@v4
# with:
# submodules: recursive
# - name: Configure
# run: |
# mkdir build && cd build
# autoreconf -i ../configure.ac
# ../configure $MPI $QUAD_P || cat config.log
# - name: Build
# run: make -C build -j
# - name: Run most tests (skip the slow ones)
# run: make -C build check-very-expensive
# - name: Save log file on failure
# uses: actions/upload-artifact@v4.4.0
# if: failure()
# with:
# name: test-suites
# path: |
# build/tests/test-suite.log
#
Loading