ReFrame 4.0.0-dev.4

Release Notes

New features and enhancements

• Build configuration incrementally by allowing to combine multiple configuration files (#2557)

Bug fixes

• Fix filtering of tests with hashcode (#2688)
• Fix variable assignment when aliases are set in subclasses (#2683)
• Fix injection of env variables for non default craype (#2682)

Test library

• Use the builtin validation option of OSU benchmarks to validate the results (#2679)

Other

• Revert workaround in script for generating release notes (#2680)
• Fix example in sn.print() docs (#2685)
• Update copyright notice in docs (#2684)

ReFrame 4.0.0-dev.3

Release Notes

New features and enhancements

• Relax the value syntax of env_vars (#2649)
• Deprecate variables in favour of the new env_vars (#2648)

Bug fixes

• Pass explicitly the --cpus-per-task option to srun for Slurm >= 22.05 (#2656)
• Expand the Slurm nodelist using scontrol and not the builtin nodelist_expand() function (#2678)
• Properly capture the node list of a job in the Slurm backend (#2634)

ReFrame 4.0.0-dev.2

Release Notes

New features and enhancements

• Introduce variable aliases (#2647)
• Change default value of num_gpus_per_node to None (#2638)
• Extend functionality to get the Cray's CDT version to support systems based on the CPE module (#2644)
• Allow tests to require references (#2627)
• Remove the deprecated RFM_GRAYLOG_SERVER environment variable (#2624)
• Improve framework messages and the performance report (#2618)

Bug fixes

• Filter out unnecessary fixtures when listing or running tests (#2641)

Test library

<n/a>

Other

• Remove mailing list from README (#2643)
• Adapt README for dark and light Github modes (#2608)

ReFrame 4.0.0-dev.1

Release Notes

New features and enhancements

• Add perflog_ignore attribute in httpjson handler (#2620)
• Allow user defined launchers in the configuration (#2605)
• Add support for the Apptainer container platform (#2613)
• Add support for the Flux scheduler (#2598)
• Stop execution with an error when a cli module fails to load (#2603)
• Revert to generating unique run reports (#2601)

Bug fixes

• Submit jobs using the correct module path when testing scheduler backends (#2611)

Test library

n/a

Other

• Fix script that auto-generates the release notes (#2594)
• Fix typo on the docs (#2599)
• Fix broken links in the docs (#2593)

ReFrame 4.0.0-dev.0

Release Notes

New features and enhancements

• Change default hostname autodetection parameters (#2583)
• Associate a unique hash with each test (#2578)
• Add command line option to set the execution order of independent tests (#2575)
• Drop the old test naming scheme (#2574)
• Introduce ReFrame 4.0 (#2562)
• Use the test's extra_resources also for the build job (#2547)
• Remove deprecated features for 4.x (#1986, #2096)
• Drop the @parameterized_test decorator (#1951)

Bug fixes

• Document missing log record attributes (#2586)
• Log correctly %(check_perfvalues)s attribute (#2588)
• Fix singularity cli option to change to the mounted stage directory with the Singularity container platform (#2581)

Test library

n/a

Other

• Fix documentation of modules test attribute (#2592)
• Add a warning in documentation about the usage of perf_patterns (#2587)
• Add CSCS webinar resources + links (#2582)
• Fix example of httpjson handler usage (#2563)
• Add note about test dependencies in --repeat option docs (#2564)

ReFrame 3.12.0

Release Notes

Changes since 3.11.0

New features and enhancements

• Add a lightweight time profiler in the framework (#2545)
• Optimise the test case generation (#2544)
• Include the loggable variables and parameters in the run report (#2543)
• Add new command line option to repeat selected tests (#2539)
• Do not require to set container_platform for running containerised tests (#2537)
• Add --mode option to GitLab CI pipeline command (#2514)

Bug fixes

• Treat undefined variables correctly in run report (#2546)
• Snooze between polls in serial policy (#2538)
• Improve test selection by their variant id (#2533)
• Reset poll rate every time a test finishes the compilation or run phase (#2534)
• Skip test file when an import error is encountered (#2535)
• Improve detection of ReFrame test files (#2520)
• Check that PBS output is written back to working directory before setting the job as completed (#2519)
• Allow setting fixture variables from the command line (#2515)

Test library

• Fix broken symlinks to the Xdevice header library (#2524)
• Make GPU burn test a library test (#2503)

Other

• Move CSCS tests and configuration out of this repo (#2521)
• Fix documentation of execution modes' target_systems (#2541)

ReFrame 3.11.2

Release Notes

New features and enhancements

n/a

Bug fixes

• Improve test selection by their variant id (#2533)
• Reset poll rate every time a test finishes the compilation or run phase (#2534)
• Skip test file when an import error is encountered (#2535)

Test library

• Fix broken symlinks to the Xdevice header library (#2524)

Other

• Move CSCS tests and configuration out of this repo (#2521)

ReFrame 3.11.1

Release Notes

New features and enhancements

• Pass --mode option to the generated GitLab CI child pipeline (#2514)

Bug fixes

• Improve detection of ReFrame test files (#2520)
• Check that PBS output is written back to working directory before setting the job as completed (#2519)
• Allow setting fixture variables from the command line (#2515)

Test library

• Make GPU burn test a library test (#2503)

Other

n/a

ReFrame 3.11.0

Release Notes

Changes since v3.10.0

New features and enhancements

• Add new command line option to distribute single node jobs on multiple cluster nodes (#2458)
• Add new build system backend for custom builds (#2494)
• Extend the syntax of valid_systems and valid_prog_environs (#2479)
• Introduce environment variables for controlling system auto-detection (#2444)
• Support early access to fixtures (#2484)
• Add configuredir variable to the Autotools build system (#2460)
• Add display name to the failure statistics (#2472)
• Extend the public API of Job (#2463)
• Add support for Python 3.10 (#2443)
• Support for loggable variables, parameters and properties of tests (#2428)
• Remove runtime timings from the default output (#2425)
• Add convenience function to set a test variable only if it's undefined (#2403)

Bug fixes

• Fix test filtering by name when multiple exact matches are requested (#2485)
• Treat empty source code context when reporting errors (#2482)
• Fix regression with tests that treat stdin as a tty (#2476)
• Make rerun prompt consistent with the new naming scheme (#2471)
• Fix and enhance LSF support (#2383)
• Properly select configuration when running tests (#2438)
• Enforce override of hooks with the same name (#2436)
• Do not set up Spack shell support (#2424)
• Fix job cancellation when job is pending due to ReqNodeNotAvail (#2417)
• Fix --show-config for null config options (#2414)
• Automatically convert numeric configuration parameters when set from the command line or the environment (#2392)
• Fix find_modules to return unique module names (#2401)
• Print skip test messages for new asynchronous execution policy (#2400)

Test library

• Add OSU benchmarks in the test library and adapt CSCS checks accordingly (#2421)
• Time the warmup phase and print minimum required duration in the GPU burn test (#2345)
• Make bench_name loggable in Gromacs library test (#2462)
• Add support for build both GPU dgemm+sgemm executables (#2372)

Other

• Use pytest-rerunfailures to and mark flaky unit tests (#2493)
• Update checkout and setup-python versions for GH actions (#2457)
• Minor deployment improvements (#2447)
• Improve documentation about setting performance references in tests (#2492)
• Modernize test syntax in the test dependencies docs (#2459)
• Update versionadded annotations (#2448)
• Add a small note in the Spack backend tutorial on specifying a spec targets (#2422)
• Use compile as the name of the build pipeline stage (#2418)
• Enhance documentation of environment variables set in the system configuration (#2419)

ReFrame 3.11.0-rc2

Release Notes

New features and enhancements

• Add display name to the failure statistics (#2472)

Bug fixes

• Fix regression with tests that treat stdin as a tty (#2476)
• Make rerun prompt consistent with the new naming scheme (#2471)

Test library

n/a

Other

n/a