Move some of the E2E tests for GEG & CEG into nightly runs #2036
+150
−53
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… tests into nightly. Move all F16 split-k tests into nightly as well except keeping one for the PR CI.
umangyadav
requested review from
Copilot,
dhernandez0,
justinrosner and
pabloantoniom
and removed request for
causten and
dhernandez0
There was a problem hiding this comment.
Pull Request Overview
Moves heavier GEG/CEG E2E tests out of PR CI into nightly and lowers workloads in PR to reduce runtime.
- Repoint F32/BF16 GEG/CEG suites to nightly by renaming directories/suite names from Pr* to non-Pr*.
- Reduce PR test sizes (notably batch size) and trim permutations.
- Add new F16 nightly suites and arch gating cfgs; update CMake to split PR vs nightly coverage.
Reviewed Changes
Copilot reviewed 18 out of 26 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| mlir/test/e2e/PrGemmElementwiseGemmF16SplitK.toml | Reduced PR test set to a smaller config to speed up CI |
| mlir/test/e2e/PrConvElementwiseGemmF16SplitK.toml | Reduced PR configs; switched to batchsize=1 |
| mlir/test/e2e/PrConvElementwiseGemmF16.toml | Simplified PR conv+gemm F16 suite to a small config |
| mlir/test/e2e/GemmElementwiseGemmF32SplitK.toml | Nightly: rename directory and suite name to non-Pr; keep config |
| mlir/test/e2e/GemmElementwiseGemmF32.toml | Nightly: rename directory and suite name to non-Pr |
| mlir/test/e2e/GemmElementwiseGemmF16SplitK.toml | New nightly F16 split-K gemm+gemm suite |
| mlir/test/e2e/GemmElementwiseGemmF16SplitK.cfg | New nightly gating for F16 split-K (requires mfma/wmma and atomic_add_f16) |
| mlir/test/e2e/GemmElementwiseGemmBF16SplitK.toml | Nightly: rename directory and suite name to non-Pr |
| mlir/test/e2e/GemmElementwiseGemmBF16.toml | Nightly: rename directory and suite name to non-Pr |
| mlir/test/e2e/GemmElementwiseGemmBF16.cfg | New nightly gating for BF16 (requires mfma/wmma) |
| mlir/test/e2e/ConvElementwiseGemmF32SplitK.toml | Nightly: rename directory; reduce workloads; suite name still “pr_*” |
| mlir/test/e2e/ConvElementwiseGemmF32.toml | Nightly: rename directory/suite name; reduce workloads |
| mlir/test/e2e/ConvElementwiseGemmF16SplitK.toml | New nightly F16 conv+gemm split-K suite |
| mlir/test/e2e/ConvElementwiseGemmF16SplitK.cfg | New nightly gating for F16 conv split-K (mfma/wmma and atomic_add_f16) |
| mlir/test/e2e/ConvElementwiseGemmF16.toml | New nightly F16 conv+gemm suite |
| mlir/test/e2e/ConvElementwiseGemmBF16SplitK.toml | Nightly: rename directory/suite name; reduce workloads |
| mlir/test/e2e/ConvElementwiseGemmBF16.toml | Nightly: rename directory/suite name; reduce workloads |
| mlir/test/e2e/CMakeLists.txt | Split PR vs nightly test lists accordingly |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
dhernandez0
reviewed
Oct 16, 2025
dhernandez0
approved these changes
Oct 16, 2025
justinrosner
approved these changes
Oct 16, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Jenkins PR CI tests are taking a lot longer right now mainly due to CEG & GEG tests. These tests happens on three different data types F32, FP16 and BF16 and also with Split-K Enabled.
Currently some of these tests are also using batch size of 64. For the testing purposes smaller batch size is sufficient.
This PR aims to lower runtime to speed up testing locally and also on Jenkins.
Technical Details
CEG/GEG tests are for three different dtypes. IMO it is sufficient to just test F16 in PR CI and keep F32 and BF16 in nightly.
Test Plan
Run both PR CI and Nightly and measure runtime for E2E tests and compare them with current runtimes. Make sure nightly doesn't timeout with these additional tests.
Test Result