ReFrame 4.9.0

Release Notes

New features

• Updates and improvements to the poll control mechanism (#3575)
• Deprecate RegressionMixin in favor of RegressionTestPlugin (#3581)
• Replace perflog_compat with perflog_multiline in logging configuration (#3578)
• Log also results from sanity-only tests through the perflog handlers (#3516)
• Support usecs (%f format) in datefmt for logging (#3570)
• Support OR-ing node states in flexible node allocation (#3569)
• Allow the -P option to redefine existing parameters (#3554)
• Facilitate local imports from configuration files (#3548)
• Add new --table-format-delim option (#3547)
• Add new --param-values-delim option (#3538)
• Add support for marking tests as expected failures (#3481)
• Enable launchers to forward environment variables (#3464)

Enhancements

• Print the performance reference tuple while running only if defined (#3517)
• Document the builtin configuration (#3558)
• Result database performance improvements (#3546)
• Add some notes and tips in the documentation of pipeline hooks (#3540)
• Add link to FOSDEM'25 presentation (#3539)
• Skip tests if their dependencies fail (#3500)
• Update Slack invite link and add a note about expiration (#3499)

Bug fixes

• Treat negative exit codes in PBS scheduler (#3571)
• Fix progress numbering for XFAIL/XPASS tests (#3529)
• Fix failure report crash in case of expected failures (#3527)

Test library

• Fix ROCm build (#3574)

Other

• Update codecov badge (#3506)
• Enable scheduler testing Github workflows on push (#3505)
• Re-enable codecov in the project (#3220)
• Restore deleted Dockerfiles (#3502)

ReFrame 4.8.4

Release Notes

Enhancements

• Documentation enhancements (#3552)
  • Add link to FOSDEM'25 presentation
  • Add some notes and tips in the docs of pipeline hooks

Bug fixes

• Fix resource leaks in osext.run_command() and in _ProcFuture (#3556)
• Do not wipe the output dir of a test if run with --dry-run (#3553)

ReFrame 4.8.3

Release Notes

Enhancements

• Show more test attributes with --describe (#3531)

Bug fixes

• Use proper locking when accessing the SQLite DB (#3533)
• Introduce optional file locking for appending to perflogs (#3523)
• Treat = properly when parameterizing tests with --parameterize (#3530)
• Properly treat forceful termination due to SIGTERM and SIGHUP (#3534)
• Disable file logging when emitting DB records (#3520)

ReFrame 4.8.2

Release Notes

Enhancements

• Improve programming error messages (#3489)

Bug fixes

• Properly cleanup build jobs on abort (#3512)
• Fix race condition in local scheduler during timeout and properly reap done processes (#3511)
• Make -n option syntax consistent across test filtering and results storage queries (#3508)
• Return Job.jobid as string always (#3493)
• Include (again) partition name in test failure info (#3487)

Other

• Bump requests package version (#3501)

ReFrame 4.8.1

Release Notes

Enhancements

• Use the --use-pep517 option in bootstrap.sh (#3478)
• Use ClusterShell to manipulate node lists instead of our own code (#3470)
• Show up front if result storage is enabled (#3472)
• Fix typo in versionadded annotation (#3473)

Bug fixes

• Use always | to join unique values in column aggregations except for --table-format=pretty (#3471)
• Unset all SBATCH_* environment variables before submitting a job (#3474)
• Use display_name when printing the test's name in FAILURE INFO (#3468)

ReFrame 4.8.0

Release Notes

New features

• Support list assignment from JSON using the -S option (#3449)
• Support discrete testcase filters when comparing performance (#3438)
• Do not emit the -lselect option if num_tasks=None (#3452)
• Allow changing the default mount location of stagedir inside containers (#3442)
• Support selecting A/B column variants in performance comparisons (#3431)
• Support reference values of zero (#3434)
• Allow scoping session-based DB queries by time (#3403)
• Store test sub-result per performance variable (#3414)
• Add support for configure directory in CMake builds (#3395)
• Enable YAML configuration files (#3370)
• Introduce %(hostname)s log format placeholder (#3346)

Enhancements

• Document the requirement of file lock support for enabling results storage (#3457)
• Add a note about results storage and analytics in README (#3453)
• Use correct OMP_PROC_BIND arguments in the tutorial examples (#3435)
• List parameterized tests in their instantiation order (#3440)
• Add warning about combining build_locally=0 and scheduler backends that cannot reliably retrieve the job's exit code (#3426)

Bug fixes

• Correctly handle verbatim scheduler options passed in the CLI in the remote auto-detection script (#3456)
• Treat correctly failed_stage=None in failure stats report (#3454)
• Fix constraint evaluation for Slurm features starting with numbers (#3448)
• Fix lock order when storing reports to the database (#3450)
• Update special parameter character in test name (#3441)
• Disallow command options from execution modes (#3432)
• Fix calculation of lower/upper reference bounds when their absolute value reaches zero (#3433)
• Insert new line between stdout/stderr listings in failures (#3429)
• Disable number parsing when in --list-stored-sessions (#3425)

Other

• Add GH action to test EOL python versions (#3417)
• Enabled slurmdbd in Docker compose setup (#3391)
• Use Python 3.13 to publish to PyPI (#3406)
• Update YAML unit test config (#3405)
• Replace symlinks with file copies in Docker compose config for Github workflows (#3396)
• Remove stale Dockerfiles (#3387)
• Update Slurm cluster images to ubuntu:22.04 (#3383)

ReFrame 4.7.4

Release Notes

Enhancements

• Fix tutorial on --performance-report default behaviour (#3389)
• Update copyright year in docs (#3384)
• Fix default value for results storage in documentation (#3368)

Bug fixes

• Fix bug in update of pipeline progress statistics (#3400)
• Do not copy or git clone sourcesdir if --dont-restage is passed (#3393, #3398)
• Properly expand environment variables in execution modes (#3390)
• Include test info when skipping tests (#3388)
• Fix assignment of default configuration parameters across systems (#3377)
• Fix ESRCH error due to race condition when spawning fast futures (#3379, #3381)

Other

• Fix wheel file pattern in the wheel validation Github action (#3401)

ReFrame 4.7.3

Release Notes

Enhancements

• Report all failed performance variables in FAILURE INFO (#3358)

Bug fixes

• Fix use of dump_pipeline_progress and RFM_DUMP_PIPELINE_PROGRESS options (#3365)
• Add missing topology_prefix option in the configuration schema (#3364)
• Check response status in the httpjson log handler and retry in case of TOO_MANY_REQUESTS (#3356)
• Treat a node in PLANNED Slurm state as available (#3361)

ReFrame 4.7.2

Release Notes

Enhancements

• Optimize JSON decoding for sessions when applying filtering criteria (#3343)

Bug fixes

• Fix treatment of Slurm constraints in job.options and system partition's access parameter (#3352)

Other

• Update GH actions for Python 3.7 after the move of runners to Ubuntu 24.04 (#3350)

ReFrame 4.7.1

Release Notes

Enhancements

• Prefix special (generated) parameters with . instead of $ (#3330)
• Update formatting of list of talks in the documentation (#3321)
• Remove tip for disabling storage results from documentation (#3319)

Bug fixes

• Exit successfully if --duration limit is reached (#3328)
• Treat correctly Slurm constrains with hyphens when filtering nodes (#3327)
• Fix nodelist abbreviation when last node number is a multiple of 10 (#3339)
• Do not ignore --filter-expr when using --describe-stored-testcases (#3329)
• Fix error in fixture resolution when multiple fixtures of the same class are instantiated with different variables (#3326)
• Fix relative imports in deeply nested tests (#3337)