Skip to content

Commit 97f5d30

Browse files
bartgolbartgol
committed
Workflows: reworked actions and ways to skip eamxx testing jobs
* Do not check labels in an action. * Implement get-labels mini-action * Add "check" jobs that, for PRs, verify that * The PR touches relevant jobs * The PR does not have "skip" labels
1 parent e902e38 commit 97f5d30

File tree

9 files changed

+239
-140
lines changed

9 files changed

+239
-140
lines changed

‎.github/actions/check-pr-paths/README.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Composite action to check if testing for this PR is relevant based on touched paths
2+
3+
This action is meant to be used inside a PR testing workflow, as
4+
5+
```yaml
6+
jobs:
7+
check_paths:
8+
steps:
9+
- name: check_relevance
10+
if: ${{ github.event_name == "pull_request" }}
11+
uses: ./.github/actions/check-pr-paths
12+
with:
13+
paths: |
14+
path1
15+
path2
16+
testing:
17+
needs: check_paths
18+
if: ${{ github.event_name != "pull_request" || needs.check_paths.outputs.touched == 'true'}}
19+
steps:
20+
- name: testing
21+
```
22+
The input `paths1 is a list of paths that the action uses to see if
23+
the PR changes any relevant file. The action sets an output `touched`,
24+
which is `true` if the PR touches any of the listed paths and `false`
25+
otherwise.
26+
27+
This action allows to skip an entire workflow, but, unlike using a 'paths' filter at the
28+
workflow trigger level, it ensures that the workflow DOES run,
29+
hence avoiding the issue where certain checks are not present on the PR.

‎.github/actions/check-pr-paths/action.yml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
name: check-pr-paths
2+
description: 'Check that the PR modifies relevant files'
3+
inputs:
4+
paths:
5+
description: 'List of paths that are considered relevant'
6+
required: true
7+
default: ''
8+
outputs:
9+
touched:
10+
description: 'Output indicating if this testing for PR is relevant'
11+
value: ${{ steps.check_paths.outputs.value }}
12+
13+
# Note: inputs are available as env vars in the shell run steps, convertet to uppercase
14+
runs:
15+
using: "composite"
16+
steps:
17+
- name: get-changed-files
18+
uses: tj-actions/changed-files@v45
19+
with:
20+
separator: ","
21+
- name: check_paths
22+
shell: bash
23+
env:
24+
CHANGED_FILES: ${{ steps.get-changed-files.outputs.all_changed_files }}
25+
run: |
26+
# Convert the input paths into an array, and create a regex from it
27+
IFS=$'\n' read -r -a paths <<< "${{ inputs.paths }}"
28+
pattern=$(IFS=\|; echo "${paths[*]}")
29+
30+
# Check for matches and echo the matching files
31+
matching_files=$(echo "$CHANGED_FILES" | grep -E "^($pattern)")
32+
if [[ -n "$matching_files" ]]; then
33+
echo "Found relevant files: "$matching_files" # Print the matching files
34+
echo "value=true" >> $GITHUB_OUTPUT
35+
else
36+
echo "No relevant files touched by this PR."
37+
echo "value=false" >> $GITHUB_OUTPUT
38+
fi
39+

‎.github/actions/check-skip-labels/README.md‎

Lines changed: 0 additions & 24 deletions
This file was deleted.

‎.github/actions/check-skip-labels/action.yml‎

Lines changed: 0 additions & 48 deletions
This file was deleted.

‎.github/actions/get-labels/README.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Composite action to retrieve labels of a PR if event is `pull_request`
2+
3+
This action is meant to be used inside a PR testing workflow, as
4+
5+
```yaml
6+
jobs:
7+
get_labels:
8+
steps:
9+
- name: get_labels
10+
uses: ./.github/actions/get-labels
11+
testing:
12+
needs: get_labels
13+
if: ${{ github.event_name != "pull_request" ||
14+
!contains(needs.get_labels.outputs.labels,"label1" ||
15+
!contains(needs.get_labels.outputs.labels,"label1" }}
16+
```
17+
The action sets the output `labels`, which is a comma-separated list
18+
containing the PR labels. If the event that triggered the workflow is
19+
not a PR, the output will be an empty string

‎.github/actions/get-labels/action.yml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: get-labels
2+
description: 'Check that the PR modifies does not have skip labels'
3+
outputs:
4+
labels:
5+
description: 'Comma-separated list of labels on the PR or empty string (if not a PR event)'
6+
value: ${{ steps.get_labels.outputs.labels }}
7+
8+
# Note: inputs are available as env vars in the shell run steps, convertet to uppercase
9+
runs:
10+
using: "composite"
11+
steps:
12+
- name: get_labels
13+
shell: sh
14+
env:
15+
PR_LABELS: ${{ toJson(github.event.pull_request.labels.*.name) }} # Pass PR label names as JSON
16+
run: |
17+
echo 'labels=\"$(echo \"$PR_LABELS\" | jq -r "join(\",\")")\"' >> $GITHUB_OUTPUT

‎.github/workflows/eamxx-sa-testing.yml‎

Lines changed: 52 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -5,17 +5,6 @@ on:
55
pull_request:
66
branches: [ master ]
77
types: [opened, synchronize, ready_for_review, reopened]
8-
paths:
9-
- components/eamxx/**
10-
- components/eam/src/physics/rrtmgp/**
11-
- components/eam/src/physics/p3/scream/**
12-
- components/eam/src/physics/cam/**
13-
- .github/workflows/eamxx-standalone-testing.yml
14-
- externals/ekat/**
15-
- externals/scorpio/**
16-
- externals/haero/**
17-
- externals/YAKL/**
18-
- components/eam/src/physics/rrtmgp/external/**
198

209
# Manual run is used to bless
2110
workflow_dispatch:
@@ -48,15 +37,51 @@ env:
4837
submit: ${{ github.event_name == 'schedule' && 'true' || 'false' }} # Submit to cdash only for nightlies
4938

5039
jobs:
40+
pr_relevant:
41+
runs-on: ubuntu-latest # This job can run anywhere
42+
outputs:
43+
value: ${{ steps.check_paths.outputs.touched }}
44+
steps:
45+
- name: check_paths
46+
if: ${{ github.event_name == "pull_request" }}
47+
uses: ./.github/actions/check-pr-relevance
48+
with:
49+
paths: |
50+
components/eamxx
51+
components/eam/src/physics/rrtmgp
52+
components/eam/src/physics/p3/scream
53+
components/eam/src/physics/cam
54+
components/eam/src/physics/rrtmgp/external
55+
externals/ekat
56+
externals/scorpio
57+
externals/haero
58+
externals/YAKL
59+
.github/workflows/eamxx-sa-testing.yml
60+
get_labels:
61+
runs-on: ubuntu-latest
62+
outputs: ${{ steps.get_labels.outputs.labels }}
63+
steps:
64+
- name: get_labels
65+
uses: ./.github/actions/get-labels
5166
gcc-openmp:
67+
needs: pr_relevant, get_labels
68+
if: ${{
69+
(github.event_name == 'pull_request' &&
70+
needs.pr_relevant.outputs.value=='true' &&
71+
!contains(needs.get_labels.outputs.labels,'CI: skip gcc') &&
72+
!contains(needs.get_labels.outputs.labels,'CI: skip openmp') &&
73+
!contains(needs.get_labels.outputs.labels,'CI: skip eamxx-sa') &&
74+
!contains(needs.get_labels.outputs.labels,'CI: skip eamxx-all')) ||
75+
(github.event_name == 'workflow_dispatch' &&
76+
github.event.inputs.job_to_run == 'gcc-openmp' ||
77+
github.event.inputs.job_to_run == 'all') ||
78+
github.event_name == 'schedule'
79+
}}
5280
runs-on: [self-hosted, ghci-snl-cpu, gcc]
5381
strategy:
5482
fail-fast: false
5583
matrix:
5684
build_type: [sp, dbg, fpe, opt]
57-
if: ${{ github.event_name != 'workflow_dispatch' ||
58-
github.event.inputs.job_to_run == 'gcc-openmp' ||
59-
github.event.inputs.job_to_run == 'all' }}
6085
name: gcc-openmp / ${{ matrix.build_type }}
6186
steps:
6287
- name: Check out the repository
@@ -67,13 +92,6 @@ jobs:
6792
submodules: recursive
6893
- name: Show action trigger
6994
uses: ./.github/actions/show-workflow-trigger
70-
- name: Check for skip labels
71-
if: ${{ github.event_name == 'pull_request' || github.event_name == 'pull_request_review' }}
72-
uses: ./.github/actions/check-skip-labels
73-
with:
74-
skip_labels: 'AT: skip gcc,AT: skip openmp,AT: skip eamxx-sa,AT: skip eamxx-all'
75-
token: ${{ secrets.GITHUB_TOKEN }}
76-
pr_number: ${{ github.event.pull_request.number }}
7795
- name: Set test-all inputs based on event specs
7896
run: |
7997
echo "generate=false" >> $GITHUB_ENV
@@ -91,14 +109,24 @@ jobs:
91109
submit: ${{ env.submit }}
92110
cmake-configs: Kokkos_ENABLE_OPENMP=ON
93111
gcc-cuda:
112+
needs: pr_relevant, get_labels
113+
if: ${{
114+
(github.event_name == 'pull_request' &&
115+
needs.pr_relevant.outputs.value=='true' &&
116+
!contains(needs.get_labels.outputs.labels,'CI: skip gcc') &&
117+
!contains(needs.get_labels.outputs.labels,'CI: skip cuda') &&
118+
!contains(needs.get_labels.outputs.labels,'CI: skip eamxx-sa') &&
119+
!contains(needs.get_labels.outputs.labels,'CI: skip eamxx-all')) ||
120+
(github.event_name == 'workflow_dispatch' &&
121+
github.event.inputs.job_to_run == 'gcc-cuda' ||
122+
github.event.inputs.job_to_run == 'all') ||
123+
github.event_name == 'schedule'
124+
}}
94125
runs-on: [self-hosted, ghci-snl-cuda, cuda, gcc]
95126
strategy:
96127
fail-fast: false
97128
matrix:
98129
build_type: [sp, dbg, opt]
99-
if: ${{ github.event_name != 'workflow_dispatch' ||
100-
github.event.inputs.job_to_run == 'gcc-cuda' ||
101-
github.event.inputs.job_to_run == 'all' }}
102130
name: gcc-cuda / ${{ matrix.build_type }}
103131
steps:
104132
- name: Check out the repository
@@ -109,13 +137,6 @@ jobs:
109137
submodules: recursive
110138
- name: Show action trigger
111139
uses: ./.github/actions/show-workflow-trigger
112-
- name: Check for skip labels
113-
if: ${{ github.event_name == 'pull_request' || github.event_name == 'pull_request_review' }}
114-
uses: ./.github/actions/check-skip-labels
115-
with:
116-
skip_labels: 'AT: skip gcc,AT: skip cuda,AT: skip eamxx-sa,AT: skip eamxx-all'
117-
token: ${{ secrets.GITHUB_TOKEN }}
118-
pr_number: ${{ github.event.pull_request.number }}
119140
- name: Set test-all inputs based on event specs
120141
run: |
121142
echo "generate=false" >> $GITHUB_ENV

‎.github/workflows/eamxx-scripts-tests.yml‎

Lines changed: 45 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,6 @@ on:
55
pull_request:
66
branches: [ master ]
77
types: [opened, synchronize, ready_for_review, reopened]
8-
paths:
9-
- components/eamxx/scripts/**
10-
- components/eamxx/cime_config/*.py
11-
- .github/workflows/eamxx-scripts-tests.yml
12-
- externals/ekat/**
13-
- externals/scorpio/**
14-
- externals/haero/**
15-
- externals/YAKL/**
16-
- components/eam/src/physics/rrtmgp/external/**
178

189
# Manual run for debug purposes only
1910
workflow_dispatch:
@@ -30,9 +21,54 @@ concurrency:
3021
cancel-in-progress: true
3122

3223
jobs:
24+
pr_relevant:
25+
if: ${{ github.event_name == 'pull_request' }}
26+
runs-on: ubuntu-latest # This job can run anywhere
27+
outputs:
28+
value: ${{ steps.check_paths.outputs.touched }}
29+
steps:
30+
- name: checkout
31+
uses: actions/checkout@v4
32+
with:
33+
submodules: false
34+
- name: check_paths
35+
uses: ./.github/actions/check-pr-paths
36+
with:
37+
paths: |
38+
components/eamxx/scripts
39+
components/eamxx/cime_config/eamxx
40+
components/eamxx/cime_config/build
41+
components/eamxx/cime_config/yaml_utils.py
42+
.github/workflows/eamxx-scripts-tests.yml
43+
get_labels:
44+
if: ${{ github.event_name == 'pull_request' }}
45+
runs-on: ubuntu-latest
46+
outputs:
47+
labels: ${{ steps.get_labels.outputs.labels }}
48+
steps:
49+
- name: checkout
50+
uses: actions/checkout@v4
51+
with:
52+
submodules: false
53+
- name: get_labels
54+
uses: ./.github/actions/get-labels
3355
cpu-gcc:
56+
needs: [pr_relevant, get_labels]
57+
# if: ${{
58+
# github.event_name != 'pull_request' ||
59+
# (
60+
# needs.pr_relevant.outputs.value == 'true'
61+
# # needs.pr_relevant.outputs.value == 'true' &&
62+
# # !contains(split(needs.get_labels.outputs.labels, ','), 'CI: skip eamxx-all')
63+
# )
64+
# }}
3465
runs-on: [self-hosted, gcc, ghci-snl-cpu]
3566
steps:
67+
- name: print-stuff
68+
run: |
69+
echo "PR Relevant Value: '${{ needs.pr_relevant.outputs.value }}'"
70+
echo "Event Name: '${{ github.event_name }}'"
71+
echo "labels: '${{ needs.get_labels.outputs.labels }}'"
3672
- name: Check out the repository
3773
uses: actions/checkout@v4
3874
with:
@@ -41,13 +77,6 @@ jobs:
4177
submodules: recursive
4278
- name: Show action trigger
4379
uses: ./.github/actions/show-workflow-trigger
44-
- name: Check for skip labels
45-
if: ${{ github.event_name == 'pull_request' || github.event_name == 'pull_request_review' }}
46-
uses: ./.github/actions/check-skip-labels
47-
with:
48-
skip_labels: 'AT: skip eamxx-all'
49-
token: ${{ secrets.GITHUB_TOKEN }}
50-
pr_number: ${{ github.event.pull_request.number }}
5180
- name: Run test
5281
run: |
5382
cd components/eamxx

0 commit comments

Comments
 (0)