Skip to content

support parallel test execution #260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

support parallel test execution #260

wants to merge 7 commits into from

Conversation

Jannik2099
Copy link
Contributor

This adds parallel test execution through the vscode TestController API. Sequential tests test(..., is_parallel = false) are respected.

Parallelism is set to the number of CPUs, if desired I could also add a config option.

I also fixed output formatting of test failures as vscode.TestRun.appendOutput always expects CLRF.

@Jannik2099
Copy link
Contributor Author

I couldn't help myself and just added test lookup functionality (right click on test -> Go to Test).

This leads to the source file that a test exe is built from. This only works for simple tests that are made up of one source file, but that should be the majority of tests.

@tristan957
Copy link
Member

@Jannik2099 do you think this PR is ready to go? Sorry it's taken me a long time to get to it

tristan957
tristan957 reviewed Oct 15, 2024
View reviewed changes
@Jannik2099 Jannik2099 force-pushed the parallel_tests branch 3 times, most recently from d70915c to ecdb6f2 Compare October 31, 2024 19:15
@Jannik2099
Copy link
Contributor Author

Sorry for the delay, I managed to break my finger which made interfacing with a keyboard... difficult.
Programming is a dangerous endeavour!

I addressed all your points.

@tristan957
Copy link
Member

I guess this slipped my mind when cutting the new release. Is there any way you could rebase on main, so I can get this for 1.28?

@Jannik2099
Copy link
Contributor Author

No worries, I rebased & tested that things still work as expected.

@tristan957
Copy link
Member

Thanks for your work. Will review next week.

@Jannik2099
Copy link
Contributor Author

ping?

tristan957
tristan957 reviewed Apr 15, 2025
View reviewed changes
Copy link
Member

@tristan957 tristan957 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial comments

src/tests.ts Outdated
Comment on lines 8 to 9
// this is far from complete, but should suffice for the
// "test is made of a single executable is made of a single source file" usecase.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you going to complete this TODO

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No.
The comment is about tests that consist of multiple TUs, or tests that are not native meson tests (see https://mesonbuild.com/Reference-manual_functions.html#test_protocol ) - I don't have a good idea on what projects using these options look like, as I personally only use single TU tests.
I'm not sure if there is a simple solution for these, and "single TU tests" should cover the majority of users.

@Jannik2099 Jannik2099 marked this pull request as draft April 16, 2025 18:22
@Jannik2099
Copy link
Contributor Author

converted back to draft as I discovered a minor issue - parallel runs could spawn parallel compile jobs.

@Jannik2099 Jannik2099 marked this pull request as ready for review April 16, 2025 20:08
tristan957
tristan957 reviewed Apr 16, 2025
View reviewed changes
Copy link
Member

@tristan957 tristan957 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting close!

@Jannik2099
allow for parallel test execution
b3b3cd8 
Signed-off-by: Jannik Glückert <jannik.glueckert@gmail.com>
@Jannik2099
fix newlines in test output
3e5deb2 
Signed-off-by: Jannik Glückert <jannik.glueckert@gmail.com>
@Jannik2099
support sequential tests in parallel executor
3265e88 
Signed-off-by: Jannik Glückert <jannik.glueckert@gmail.com>
@Jannik2099
add rudimentary test -> test source lookup
ebccace 
Signed-off-by: Jannik Glückert <jannik.glueckert@gmail.com>
@Jannik2099
use full target name in testDebugHandler
79f6cef 
this is required when targets exist twice,
e.g. due to subprojects

Signed-off-by: Jannik Glückert <jannik.glueckert@gmail.com>
@Jannik2099
make number of test parallelism configurable
83893b0 
Signed-off-by: Jannik Glückert <jannik.glueckert@gmail.com>
@Jannik2099 Jannik2099 mentioned this pull request Apr 18, 2025
Open
@Jannik2099
gather all tests for rebuild before issuing execution
93ad6d6 
otherwise multiple simultaneous test runs may trigger rebuilds

Signed-off-by: Jannik Glückert <jannik.glueckert@gmail.com>
@Jannik2099
Copy link
Contributor Author

ping?

@tristan957
Copy link
Member

Sorry, I haven't had much time to look at your PRs. I promise to get to them soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tristan957 tristan957 tristan957 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Jannik2099 @tristan957