Enable use of a unit test thread limit in Jenkins. #39481
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adding this to the run-tests script will allow a per-node configuration of the maximum number of units tests to run in parallel.
thomashampson
added
High Priority
jhaigh0
approved these changes
Jun 13, 2025
peterfpeterson
pushed a commit
to peterfpeterson/mantid
that referenced
this pull request
Jun 20, 2025
Add a check for a new `UNIT_TEST_THREADS` environment variable that can be set in the Jenkins configuration for each node that can be used to limit the number of unit tests to be run in parallel. This has been done so that we can limit the number of unit tests processes on out macOS builders, some of which have limited RAM (compared to the core count). ### Further detail of work I have set `UNIT_TEST_THREADS=6` on the following nodes, which have been observed to timeout on tests: - isis-ndw2564 - isis-ndw2995 - isis-ndw2996 This value is two below the number of cores, so will hopefully give the machines more resources to run the tests in parallel successfully. We can tweak the number inside the Jenkins configuration if required.
peterfpeterson
pushed a commit
to peterfpeterson/mantid
that referenced
this pull request
Jun 20, 2025
Add a check for a new `UNIT_TEST_THREADS` environment variable that can be set in the Jenkins configuration for each node that can be used to limit the number of unit tests to be run in parallel. This has been done so that we can limit the number of unit tests processes on out macOS builders, some of which have limited RAM (compared to the core count). ### Further detail of work I have set `UNIT_TEST_THREADS=6` on the following nodes, which have been observed to timeout on tests: - isis-ndw2564 - isis-ndw2995 - isis-ndw2996 This value is two below the number of cores, so will hopefully give the machines more resources to run the tests in parallel successfully. We can tweak the number inside the Jenkins configuration if required.
peterfpeterson
added a commit
that referenced
this pull request
Jun 20, 2025
This pulls a variety of changes to buildscripts, dependencies, and assundries upstream of those changes. * #39108 * #39440 * #38976 * #39240 * #39522 * #39337 * #39481 * #39295 * #39360 * #39355 * #39170 * #39343 * #37350 --------- Co-authored-by: Darsh Dinger <106342815+darshdinger@users.noreply.github.com> Co-authored-by: Jack <jack.allen@stfc.ac.uk> Co-authored-by: thomashampson <thomas.hampson@stfc.ac.uk> Co-authored-by: James Clarke <139879523+jclarkeSTFC@users.noreply.github.com> Co-authored-by: Sarah Foxley <55837273+sf1919@users.noreply.github.com> Co-authored-by: Adam J. Jackson <a.j.jackson@physics.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of work
Add a check for a new
UNIT_TEST_THREADSenvironment variable that can be set in the Jenkins configuration for each node that can be used to limit the number of unit tests to be run in parallel. This has been done so that we can limit the number of unit tests processes on out macOS builders, some of which have limited RAM (compared to the core count).Further detail of work
I have set
UNIT_TEST_THREADS=6on the following nodes, which have been observed to timeout on tests:This value is two below the number of cores, so will hopefully give the machines more resources to run the tests in parallel successfully. We can tweak the number inside the Jenkins configuration if required.
Purpose of work
Often the nightly will fail because unit tests have timed out on macOS.
Fixes #39434 (hopefully!)
To test:
Search the osx build log for this PR for
ctest -j. If it is one of the machines listed above, it should have a 6 as the next character.This does not require release notes because it is a build configuration change.
Reviewer
Please comment on the points listed below (full description).
Your comments will be used as part of the gatekeeper process, so please comment clearly on what you have checked during your review. If changes are made to the PR during the review process then your final comment will be the most important for gatekeepers. In this comment you should make it clear why any earlier review is still valid, or confirm that all requested changes have been addressed.
Code Review
Functional Tests
Does everything look good? Mark the review as Approve. A member of
@mantidproject/gatekeeperswill take care of it.Gatekeeper
If you need to request changes to a PR then please add a comment and set the review status to "Request changes". This will stop the PR from showing up in the list for other gatekeepers.