Skip to content

Commit 9b76359

Browse files
authored
chore(dev): fix incorrect automated benchmark threshold handling (#1046)
A bug in the benchmarking report triggered a fail if the benchmark of the branch to merge scored more than the threshold over the main branch (ie. the merge branch is better), where it should fail if the score is more than the threshold below the main branch. This PR aims to fix that, and also improves reporting in case of a fail. Kind regards, Hans
1 parent 9063215 commit 9b76359

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

benchmarks/report.js

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -127,8 +127,8 @@ function reportAverages (avg) {
127127
const correctedDiff = unit === 'ms' ? diff * -1 : diff
128128
const sign = correctedDiff > 0 ? '+' : ''
129129
const perc = correctedDiff === 0 ? 100 : `${sign}${correctedDiff.toFixed(2)}`
130-
if (diff > threshold) {
131-
console.error(`\n\nError: ${key} is more than ${sign}${threshold}% off the reference (${ref} ${unit})`)
130+
if ((-1 * correctedDiff) > threshold) {
131+
console.error(`\n\nError: ${key} is ${sign}${correctedDiff.toFixed(2)}% off the reference (${ref} ${unit}) which is more than the threshold of ${threshold}% `)
132132
failed = true
133133
}
134134
console.log(`| ${label} | ${benchmark} | ${config} | ${value.toFixed(0)} | ${unit} | ${perc}%`)

0 commit comments

Comments
 (0)