Skip to content

Commit d06504a

Browse files
WeiqunZhangWeiqunZhang
authored
Simplify cache cleanup actions (#4121)
In our old approach, a workflow file contains a job that uploads the PR number as an artifact. While the PR is still open, the workflow_run triggered by it will download the artifact and use the information to clean up all except the last used cache associated with that original workflow. When a PR is merged or closed, there will be a post-pr workflow that uploads the PR number as an artifact and triggers a workflow_run that clean up all caches associated with the PR. The reason we did it this way was in the cache cleanup workflows, we did not find an easy way to get the number of the PR triggering them. This is not convenient because we have to add jobs uploading artifacts to workflow files. After some experiments, we have found a reliable way to find the PR number without using artifacts. The workflow_run's payload always contains the head SHA of the commit that triggers it, whether the PR comes from a fork or not. We can then use `gh pr list` to search for that head and obtain the PR number.
1 parent 7c2ef81 commit d06504a

17 files changed

+18
-224
lines changed

‎.github/workflows/apps.yml‎

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -152,18 +152,3 @@ jobs:
152152
153153
ccache -s
154154
du -hs ~/.cache/ccache
155-
156-
save_pr_number:
157-
if: github.event_name == 'pull_request'
158-
runs-on: ubuntu-latest
159-
steps:
160-
- name: Save PR number
161-
env:
162-
PR_NUMBER: ${{ github.event.number }}
163-
run: |
164-
echo $PR_NUMBER > pr_number.txt
165-
- uses: actions/upload-artifact@v4
166-
with:
167-
name: pr_number
168-
path: pr_number.txt
169-
retention-days: 1

‎.github/workflows/bittree.yml‎

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -110,18 +110,3 @@ jobs:
110110
111111
ccache -s
112112
du -hs ~/.cache/ccache
113-
114-
save_pr_number:
115-
if: github.event_name == 'pull_request'
116-
runs-on: ubuntu-latest
117-
steps:
118-
- name: Save PR number
119-
env:
120-
PR_NUMBER: ${{ github.event.number }}
121-
run: |
122-
echo $PR_NUMBER > pr_number.txt
123-
- uses: actions/upload-artifact@v4
124-
with:
125-
name: pr_number
126-
path: pr_number.txt
127-
retention-days: 1

‎.github/workflows/clang.yml‎

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -212,18 +212,3 @@ jobs:
212212
213213
ccache -s
214214
du -hs ~/.cache/ccache
215-
216-
save_pr_number:
217-
if: github.event_name == 'pull_request'
218-
runs-on: ubuntu-latest
219-
steps:
220-
- name: Save PR number
221-
env:
222-
PR_NUMBER: ${{ github.event.number }}
223-
run: |
224-
echo $PR_NUMBER > pr_number.txt
225-
- uses: actions/upload-artifact@v4
226-
with:
227-
name: pr_number
228-
path: pr_number.txt
229-
retention-days: 1

‎.github/workflows/cleanup-cache-postpr.yml‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,11 @@ jobs:
2323
2424
REPO=${{ github.repository }}
2525
26-
gh run download ${{ github.event.workflow_run.id }} -n pr_number
27-
pr_number=`cat pr_number.txt`
26+
# For debugging cat ${GITHUB_EVENT_PATH} to see the payload.
27+
28+
pr_head_sha=${{ github.event.workflow_run.head_sha }}
29+
pr_number=$(gh pr list --state all --search $pr_head_sha --json number --jq '.[0].number')
30+
echo "Post-PR cache cleanup for PR ${pr_number}"
2831
BRANCH=refs/pull/${pr_number}/merge
2932
3033
# Setting this to not fail the workflow while deleting cache keys.

‎.github/workflows/cleanup-cache.yml‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,9 +29,12 @@ jobs:
2929
# Triggering workflow run name (e.g., LinuxClang)
3030
WORKFLOW_NAME="${{ github.event.workflow_run.name }}"
3131
32+
# For debugging, cat ${GITHUB_EVENT_PATH} to see the payload.
33+
3234
if [[ $EVENT == "pull_request" ]]; then
33-
gh run download ${{ github.event.workflow_run.id }} -n pr_number
34-
pr_number=`cat pr_number.txt`
35+
pr_head_sha=${{ github.event.workflow_run.head_sha }}
36+
pr_number=$(gh pr list --search $pr_head_sha --json number --jq '.[0].number')
37+
echo "Clean up cache for PR ${pr_number}"
3538
BRANCH=refs/pull/${pr_number}/merge
3639
else
3740
BRANCH=refs/heads/${{ github.event.workflow_run.head_branch }}
@@ -54,6 +57,7 @@ jobs:
5457
IFS=$'\n'
5558
for j in $cached_jobs
5659
do
60+
# Delete all entries except the last used one
5761
old_keys=$(gh actions-cache list -L 100 -R $REPO -B $BRANCH --key "${j}-git-" --sort last-used | cut -f 1 | tail -n +2)
5862
for k in $old_keys
5963
do

‎.github/workflows/codeql.yml‎

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -92,18 +92,3 @@ jobs:
9292
uses: github/codeql-action/analyze@v3
9393
with:
9494
category: "/language:${{ matrix.language }}"
95-
96-
save_pr_number:
97-
if: github.event_name == 'pull_request'
98-
runs-on: ubuntu-latest
99-
steps:
100-
- name: Save PR number
101-
env:
102-
PR_NUMBER: ${{ github.event.number }}
103-
run: |
104-
echo $PR_NUMBER > pr_number.txt
105-
- uses: actions/upload-artifact@v4
106-
with:
107-
name: pr_number
108-
path: pr_number.txt
109-
retention-days: 1

‎.github/workflows/cuda.yml‎

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -205,18 +205,3 @@ jobs:
205205
206206
ccache -s
207207
du -hs ~/.cache/ccache
208-
209-
save_pr_number:
210-
if: github.event_name == 'pull_request'
211-
runs-on: ubuntu-latest
212-
steps:
213-
- name: Save PR number
214-
env:
215-
PR_NUMBER: ${{ github.event.number }}
216-
run: |
217-
echo $PR_NUMBER > pr_number.txt
218-
- uses: actions/upload-artifact@v4
219-
with:
220-
name: pr_number
221-
path: pr_number.txt
222-
retention-days: 1

‎.github/workflows/gcc.yml‎

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -664,18 +664,3 @@ jobs:
664664
mpirun -np 2 ./main3d.gnu.TPROF.MPI.ex ./inputs
665665
h5dump -d "level_0/data:offsets=0" -s "1" -c "1" ./plt00000.h5
666666
h5dump -d "level_0/data:datatype=1" -s "1" -c "1" ./plt00000/particle0/particle0.h5
667-
668-
save_pr_number:
669-
if: github.event_name == 'pull_request'
670-
runs-on: ubuntu-latest
671-
steps:
672-
- name: Save PR number
673-
env:
674-
PR_NUMBER: ${{ github.event.number }}
675-
run: |
676-
echo $PR_NUMBER > pr_number.txt
677-
- uses: actions/upload-artifact@v4
678-
with:
679-
name: pr_number
680-
path: pr_number.txt
681-
retention-days: 1

‎.github/workflows/hip.yml‎

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -181,18 +181,3 @@ jobs:
181181
182182
ccache -s
183183
du -hs ~/.cache/ccache
184-
185-
save_pr_number:
186-
if: github.event_name == 'pull_request'
187-
runs-on: ubuntu-latest
188-
steps:
189-
- name: Save PR number
190-
env:
191-
PR_NUMBER: ${{ github.event.number }}
192-
run: |
193-
echo $PR_NUMBER > pr_number.txt
194-
- uses: actions/upload-artifact@v4
195-
with:
196-
name: pr_number
197-
path: pr_number.txt
198-
retention-days: 1

‎.github/workflows/hypre.yml‎

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -154,18 +154,3 @@ jobs:
154154
155155
ccache -s
156156
du -hs ~/.cache/ccache
157-
158-
save_pr_number:
159-
if: github.event_name == 'pull_request'
160-
runs-on: ubuntu-latest
161-
steps:
162-
- name: Save PR number
163-
env:
164-
PR_NUMBER: ${{ github.event.number }}
165-
run: |
166-
echo $PR_NUMBER > pr_number.txt
167-
- uses: actions/upload-artifact@v4
168-
with:
169-
name: pr_number
170-
path: pr_number.txt
171-
retention-days: 1

0 commit comments

Comments
 (0)