-
-
Notifications
You must be signed in to change notification settings - Fork 235
feat: automate benchmark on PR requests using Github Actions #1037
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: automate benchmark on PR requests using Github Actions #1037
Conversation
Pull Request Test Coverage Report for Build 16285704114Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So for now I used Github Job Summary instead.
Ok!
@seriousme I'm wondering also about adding an overall I think that we could consider there is a performance impact when we see performances below 10% |
@seriousme Will do the major bump once we merge this one :) |
Do you also want to migrate to ESM before the major? |
Could make sense, yes. Since nodejs added the support for require of esm modules it's not a pain now, just be sure we not use top level await |
|
@seriousme right, sorry, I always forget this package doesn't have it |
This PR adds Github actions to automate benchmark running on PR requests
It will create a Github Job Summary of the benchmark and add it to the benchmark action (see https://github.com/moscajs/aedes/actions/runs/16239498783#summary-45853979456 )
The benchmark steps run serialized in the hope that performance on a single runner does not vary too much.
I have been building testing this using seriousme#1 which also shows how it operates.
It also shows that variation between results is inevitable, even when running on the same runner and the Aedes code is identical !!
In seriousme#1 I was able to add the comment automatically to the PR using
.github/actions/sticky-pr-comment/action.yml
however if you send a PR from a fork the GITHUB_TOKEN that runs the actions by default (event: pull_request ) only hasread
access on the PR for security reasons (see Pull Request events for forked repositories)This can be fixed by attaching to the event: pull_request_target, however Github notes explicitly: Avoid using this event if you need to build or run code from the pull request.
So for now I used Github Job Summary instead.
Kind regards,
Hans