-
Notifications
You must be signed in to change notification settings - Fork 977
Use peak memory usage as a better proxy for ctest parallelism #18603
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
bdice
wants to merge
10
commits into
rapidsai:branch-25.12
Choose a base branch
from
bdice:ctest-parallelism
base: branch-25.12
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+56
−36
Open
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
0d894ef
Use peak memory usage as a better proxy for ctest parallelism
bdice 1dadd4b
Use 100% for LARGE_STRINGS_TEST
bdice 527535f
Try allowing 14 tests to run simultaneously.
bdice 4de4c8b
Try 10%
bdice 6f88b62
Try 15% default, 70% for most of the large tests.
bdice 58b10f0
Use async and 14-way parallelism for small tests.
bdice 6f2e3b1
Output tests using more than 1 GB of memory.
bdice 7cbe98a
Add note about memory usage
bdice ceb31f9
Update PERCENT 100 based on memory usage
bdice 40b0a9d
Rerun CI
bdice File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -4,13 +4,32 @@ | |||||
export GTEST_CUDF_RMM_MODE=cuda | ||||||
export GTEST_CUDF_MEMORY_PEAK=1 | ||||||
export GTEST_BRIEF=1 | ||||||
|
||||||
# Collect all test results | ||||||
results=() | ||||||
for gt in gtests/*_TEST ; do | ||||||
test_name=$(basename "${gt}") | ||||||
echo -n "$test_name" | ||||||
# dependent on the string output from testing_main.hpp | ||||||
bytes=$(${gt} 2>/dev/null | grep Peak | cut -d' ' -f4) | ||||||
echo ",${bytes}" | ||||||
results+=("$test_name,$bytes") | ||||||
done | ||||||
|
||||||
unset GTEST_BRIEF | ||||||
unset GTEST_CUDF_RMM_MODE | ||||||
unset GTEST_CUDF_MEMORY_PEAK | ||||||
|
||||||
# Output tests using more than 1GB | ||||||
echo "" | ||||||
echo "Tests using more than 1GB of memory:" | ||||||
threshold=1073741824 # 1GB in bytes | ||||||
for result in "${results[@]}" ; do | ||||||
test_name=$(echo "$result" | cut -d',' -f1) | ||||||
bytes=$(echo "$result" | cut -d',' -f2) | ||||||
if [[ -n "$bytes" && "$bytes" -gt "$threshold" ]] ; then | ||||||
# Convert bytes to GB with 2 decimal places | ||||||
gb=$(awk "BEGIN {printf \"%.2f\", $bytes / 1073741824}") | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
echo "$test_name: ${gb} GB" | ||||||
fi | ||||||
done |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.