Skip to content

Conversation

michaelmckinsey1
Copy link
Collaborator

@michaelmckinsey1 michaelmckinsey1 commented Jul 11, 2025

Description

  • Currently our codecoverage is only tracking the pytest testing, which is a subset of our CI. This PR adds additional reporting to codecoverage for:
    • GitHub dryruns (8x matrix runs)
    • Saxpy binary execution test (1x)
  • Coverage boosted by +23%
  • Depends on Automatically Enumerate Dryruns and MpiOnlyExperiment #904
  • There would be an additional boost if we could figure this out for gitlab tests, but this would be a big lift.

Adding/modifying core functionality, CI, or documentation:

  • Update bin/benchpark to run coverage if environment variable BENCHPARK_RUN_COVERAGE is set
  • Update .github and `codecov.yml to implement and support extra coverage uploads
  • Update dev docs to explain coverage

@github-actions github-actions bot added feature New feature or request ci CI, unit tests, GitHub actions labels Jul 11, 2025
@github-actions github-actions bot added the dependencies Pull requests that update a dependency file label Jul 11, 2025
@michaelmckinsey1 michaelmckinsey1 marked this pull request as draft July 11, 2025 22:49
@michaelmckinsey1 michaelmckinsey1 marked this pull request as ready for review July 11, 2025 22:49
@michaelmckinsey1 michaelmckinsey1 self-assigned this Jul 14, 2025
@michaelmckinsey1 michaelmckinsey1 changed the title [WIP] codecov testing [WIP] Optionally Generate Coverage from Any Usage of bin/benchpark Jul 31, 2025
@slabasan slabasan added the WIP A work-in-progress not yet ready to commit label Aug 1, 2025
@michaelmckinsey1
Copy link
Collaborator Author

Ready if passing CI

@michaelmckinsey1 michaelmckinsey1 added ready for review Ready for review and removed WIP A work-in-progress not yet ready to commit labels Aug 4, 2025
@michaelmckinsey1 michaelmckinsey1 changed the title [WIP] Optionally Generate Coverage from Any Usage of bin/benchpark Additionally Generate Coverage from Any Usage of bin/benchpark Aug 4, 2025
@github-actions github-actions bot added the docs Improvements or additions to documentation label Aug 4, 2025
@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.44%. Comparing base (2e52600) to head (3e12646).
⚠️ Report is 3 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff              @@
##           develop     #891       +/-   ##
============================================
+ Coverage    41.75%   65.44%   +23.69%     
============================================
  Files           24       45       +21     
  Lines         1583     3094     +1511     
  Branches       231      231               
============================================
+ Hits           661     2025     +1364     
- Misses         872     1061      +189     
+ Partials        50        8       -42     

see 33 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@michaelmckinsey1 michaelmckinsey1 changed the title Additionally Generate Coverage from Any Usage of bin/benchpark Additionally Generate Coverage from Any Usage of bin/benchpark if Environment Variable Provided Aug 5, 2025
@slabasan slabasan added this pull request to the merge queue Aug 5, 2025
Merged via the queue into develop with commit 92c7cbc Aug 5, 2025
47 checks passed
@slabasan slabasan deleted the codecov-testing branch August 5, 2025 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci CI, unit tests, GitHub actions dependencies Pull requests that update a dependency file docs Improvements or additions to documentation feature New feature or request ready for review Ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants