Skip to content

Conversation

@jrfnl
Copy link
Member

@jrfnl jrfnl commented Sep 17, 2025

GH Actions: "pin" all action runners

Recently there has been more and more focus on securing GH Actions workflows - in part due to some incidents.

The problem with "unpinned" action runners is as follows:

  • Tags are mutable, which means that a tag could point to a safe commit today, but to a malicious commit tomorrow.
    Note that GitHub is currently beta-testing a new "immutable releases" feature (= tags and release artifacts can not be changed anymore once the release is published), but whether that has much effect depends on the ecosystem of the packages using the feature.
    Aside from that, it will likely take years before all projects adopt immutable releases.
  • Action runners often don't even point to a tag, but to a branch, making the used action runner a moving target.
    Note: this type of "floating major" for action runners used to be promoted as good practice when the ecosystem was "young". Insights have since changed.

While it is convenient to use "floating majors" of action runners, as this means you only need to update the workflows on a new major release of the action runner, the price is higher risk of malicious code being executed in workflows.

Dependabot, by now, can automatically submit PRs to update pinned action runners too, as long as the commit-hash pinned runner is followed by a comment listing the released version the commit is pointing to.

So, what with Dependabot being capable of updating workflows with pinned action runners, I believe it is time to update the workflows to the current best practice of using commit-hash pinned action runners.

The downside of this change is that there will be more frequent Dependabot PRs.

If this would become a burden/irritating, the following mitigations can be implemented:

  1. Updating the Dependabot config to group updates instead of sending individual PRs per action runner.
  2. A workflow to automatically merge Dependabot PRs as long as CI passes.

Ref: https://docs.github.com/en/actions/reference/security/secure-use#using-third-party-actions

Dependabot: update config

This commit makes two changes to the Dependabot config:

  1. It introduces a "cooldown" period for updates to a new major release of action runners.
    What this means, is that for updates to a new major, the Dependabot will be delayed by 10 days, which should give projects the chance to fix any "teething problems".
  2. It introduces a "group".
    By default Dependabot raises individual PRs for each update. Now, it will group updates to new minor or patch release for all action runners into a single PR.
    Updates to new major releases of action runners will still be raised as individual PRs.

Refs:

Recently there has been more and more focus on securing GH Actions workflows - in part due to some incidents.

The problem with "unpinned" action runners is as follows:
* Tags are mutable, which means that a tag could point to a safe commit today, but to a malicious commit tomorrow.
    Note that GitHub is currently beta-testing a new "immutable releases" feature (= tags and release artifacts can not be changed anymore once the release is published), but whether that has much effect depends on the ecosystem of the packages using the feature.
    Aside from that, it will likely take years before all projects adopt _immutable releases_.
* Action runners often don't even point to a tag, but to a branch, making the used action runner a moving target.
    _Note: this type of "floating major" for action runners used to be promoted as good practice when the ecosystem was "young". Insights have since changed._

While it is convenient to use "floating majors" of action runners, as this means you only need to update the workflows on a new major release of the action runner, the price is higher risk of malicious code being executed in workflows.

Dependabot, by now, can automatically submit PRs to update pinned action runners too, as long as the commit-hash pinned runner is followed by a comment listing the released version the commit is pointing to.

So, what with Dependabot being capable of updating workflows with pinned action runners, I believe it is time to update the workflows to the _current_ best practice of using commit-hash pinned action runners.

The downside of this change is that there will be more frequent Dependabot PRs.

If this would become a burden/irritating, the following mitigations can be implemented:
1. Updating the Dependabot config to group updates instead of sending individual PRs per action runner.
2. A workflow to automatically merge Dependabot PRs as long as CI passes.

Ref: https://docs.github.com/en/actions/reference/security/secure-use#using-third-party-actions
This commit makes two changes to the Dependabot config:
1. It introduces a "cooldown" period for updates to a new major release of action runners.
    What this means, is that for updates to a new major, the Dependabot will be delayed by 10 days, which should give projects the chance to fix any "teething problems".
2. It introduces a "group".
    By default Dependabot raises individual PRs for each update. Now, it will group updates to new minor or patch release for all action runners into a single PR.
    Updates to new major releases of action runners will still be raised as individual PRs.

Refs:
* https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/optimizing-pr-creation-version-updates
* https://docs.github.com/en/code-security/dependabot/working-with-dependabot/dependabot-options-reference
@jrfnl jrfnl added this to the 1.x Next milestone Sep 17, 2025
@jrfnl jrfnl merged commit e692881 into stable Sep 17, 2025
86 checks passed
@jrfnl jrfnl deleted the feature/ghactions-pin-action-runners branch September 17, 2025 04:26
@theofidry
Copy link

Action runners often don't even point to a tag, but to a branch

...

sight OK time to change it everywhere...

@jrfnl
Copy link
Member Author

jrfnl commented Sep 17, 2025

@theofidry Exactly ;-)

@jrfnl
Copy link
Member Author

jrfnl commented Sep 17, 2025

@theofidry FYI: there are some tools which can help with this change-over, then again, you may have found those already ? Let me know if you want some pointers.

@theofidry
Copy link

I was planning to give a go at https://github.com/suzuki-shunsuke/pinact but if you have recommendations I'm gladly pick them!

@jrfnl
Copy link
Member Author

jrfnl commented Sep 17, 2025

I was planning to give a go at https://github.com/suzuki-shunsuke/pinact but if you have recommendations I'm gladly pick them!

Ha, you found one already ;-)
Yes, that's the one I used to create the first commit in this PR.

I'd also like to run it in CI to prevent unpinned action runners being re-introduced in the workflows and while they do offer an action for this, that action cannot do the "does this commit hash actually match the version listed in the comment" verification (yet) (issue), while I would like to ensure that check is also run, so I'm still doing some more investigating to get a reusable workflow for that set up.

As I'm basically auditing workflow security now, I'm also running zizmor over each repo.
Example of one the PRs resulting from that: #198

Zizmor gives some false positives in my experience so far, especially for pull_request_target workflows which don't do a repo checkout and sometimes also about secure handling of inputs, but better to be warned and dismiss something as invalid than to miss it.

Zizmor also helps point out which workflows need better permission settings, but figuring out what the permissions should be isn't always straight-forward.
This tool can help with that: https://github.com/GitHubSecurityLab/actions-permissions/tree/main/monitor but I don't really want to leave that in production workflows for too long, so I'm also looking at the knowledge base from step-security to gather information.

Hope this helps. I'm still reviewing some other tools as well, but the above are my findings about tooling usefulness so far ;-)

@jrfnl
Copy link
Member Author

jrfnl commented Sep 17, 2025

Oh and I should probably also mention that I already run yamllint and actionlint on all repos too (at least in this organisation).
See this re-usable workflow: https://github.com/PHPCSStandards/.github/blob/main/.github/workflows/reusable-yamllint.yml

@jrfnl
Copy link
Member Author

jrfnl commented Sep 18, 2025

@theofidry Just noticed one of the pinact updates wasn't done in the way it was intended, so careful review of the output is still warranted.
See: #199

@jrfnl
Copy link
Member Author

jrfnl commented Sep 19, 2025

@theofidry FYI: #200

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants