Skip to content

Conversation

seriousme
Copy link
Contributor

@seriousme seriousme commented Jul 12, 2025

This PR adds Github actions to automate benchmark running on PR requests

It will create a Github Job Summary of the benchmark and add it to the benchmark action (see https://github.com/moscajs/aedes/actions/runs/16239498783#summary-45853979456 )

The benchmark steps run serialized in the hope that performance on a single runner does not vary too much.
I have been building testing this using seriousme#1 which also shows how it operates.

It also shows that variation between results is inevitable, even when running on the same runner and the Aedes code is identical !!

In seriousme#1 I was able to add the comment automatically to the PR using .github/actions/sticky-pr-comment/action.yml however if you send a PR from a fork the GITHUB_TOKEN that runs the actions by default (event: pull_request ) only has read access on the PR for security reasons (see Pull Request events for forked repositories)

This can be fixed by attaching to the event: pull_request_target, however Github notes explicitly: Avoid using this event if you need to build or run code from the pull request.

So for now I used Github Job Summary instead.

Kind regards,
Hans

@coveralls
Copy link

coveralls commented Jul 12, 2025

Pull Request Test Coverage Report for Build 16285704114

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 99.572%

Totals Coverage Status
Change from base Build 16267259002: 0.0%
Covered Lines: 827
Relevant Lines: 828

💛 - Coveralls

Copy link
Member

@robertsLando robertsLando left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So for now I used Github Job Summary instead.

Ok!

@robertsLando
Copy link
Member

@seriousme I'm wondering also about adding an overall % for all tests, an average of the results, like -3%.

I think that we could consider there is a performance impact when we see performances below 10%

@robertsLando
Copy link
Member

@seriousme Will do the major bump once we merge this one :)

@seriousme
Copy link
Contributor Author

Do you also want to migrate to ESM before the major?

@robertsLando
Copy link
Member

Do you also want to migrate to ESM before the major?

Could make sense, yes. Since nodejs added the support for require of esm modules it's not a pain now, just be sure we not use top level await

@seriousme
Copy link
Contributor Author

npm: cache only works if package-lock.json is present

@robertsLando
Copy link
Member

@seriousme right, sorry, I always forget this package doesn't have it

@robertsLando robertsLando merged commit 86f5b45 into moscajs:main Jul 15, 2025
13 checks passed
@seriousme seriousme deleted the github-action-benchmark-compare branch July 15, 2025 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants