Skip to content

Failed Test Replay #9214

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

Failed Test Replay #9214

wants to merge 7 commits into from

Conversation

daniel-mohedano
Copy link
Contributor

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@daniel-mohedano daniel-mohedano added type: enhancement Enhancements and improvements tag: do not merge Do not merge changes comp: ci visibility Continuous Integration Visibility comp: debugger Dynamic Instrumentation labels Jul 22, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jul 23, 2025

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1753281005 1753281351
end_time 2025-07-23T14:31:27 2025-07-23T14:37:12
git_branch master daniel.mohedano/failed-test-replay
git_commit_sha 6fc7077 c7eeac8
start_time 2025-07-23T14:30:06 2025-07-23T14:35:52
See matching parameters
Baseline Candidate
ci_job_id 1043474975 1043474975
ci_pipeline_id 71456628 71456628
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1753280286 1753280286

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 6 unstable metrics.

See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-45.234µs; +16.922µs] or [-15.606%; +5.838%]		 unstable
[-61.872µs; +23.251µs] or [-18.578%; +6.982%]		 unstable
[-72.622µs; +32.029µs] or [-20.886%; +9.212%]		 unstable
[-71.732µs; +265.705µs] or [-7.563%; +28.015%]		 same
scenario:basic same same same unstable
[+80.247µs; +390.785µs] or [+11.309%; +55.073%]		 unstable
[-178.550op/s; +178.550op/s] or [-6.785%; +6.785%]
scenario:loop unsure
[+2.937µs; +12.727µs] or [+0.033%; +0.143%]		 unsure
[+5.060µs; +16.885µs] or [+0.056%; +0.188%]		 unsure
[+3.988µs; +16.887µs] or [+0.044%; +0.187%]		 same same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (333.032 µs) : 281, 385
.   : milestone, 333,
basic (281.157 µs) : 274, 289
.   : milestone, 281,
loop (8.965 ms) : 8960, 8971
.   : milestone, 8965,
section candidate
noprobe (313.722 µs) : 290, 337
.   : milestone, 314,
basic (278.895 µs) : 273, 284
.   : milestone, 279,
loop (8.976 ms) : 8971, 8982
.   : milestone, 8976,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 333.032 µs [280.822 µs, 385.242 µs]
basic 281.157 µs [273.548 µs, 288.766 µs]
loop 8.965 ms [8.96 ms, 8.971 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 313.722 µs [290.147 µs, 337.297 µs]
basic 278.895 µs [273.342 µs, 284.447 µs]
loop 8.976 ms [8.971 ms, 8.982 ms]

@pr-commenter
Copy link

pr-commenter bot commented Jul 23, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master daniel.mohedano/failed-test-replay
git_commit_date 1753268392 1753280286
git_commit_sha 6fc7077 c7eeac8
release_version 1.51.1-SNAPSHOT~6fc7077594 1.51.1-SNAPSHOT~c7eeac8854
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1753282264 1753282264
ci_job_id 1043474968 1043474968
ci_pipeline_id 71456628 71456628
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-3l8xp6vh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-3l8xp6vh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 11 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.51.1-SNAPSHOT~c7eeac8854, baseline=1.51.1-SNAPSHOT~6fc7077594

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1058437
Total [baseline] (8.625 s) : 0, 8625336
Agent [candidate] (1.04 s) : 0, 1040187
Total [candidate] (8.625 s) : 0, 8624580
section iast
Agent [baseline] (1.179 s) : 0, 1178674
Total [baseline] (9.303 s) : 0, 9302919
Agent [candidate] (1.187 s) : 0, 1187324
Total [candidate] (9.311 s) : 0, 9311483
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.058 s -
Agent iast 1.179 s 120.237 ms (11.4%)
Total tracing 8.625 s -
Total iast 9.303 s 677.583 ms (7.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.04 s -
Agent iast 1.187 s 147.137 ms (14.1%)
Total tracing 8.625 s -
Total iast 9.311 s 686.903 ms (8.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.51.1-SNAPSHOT~c7eeac8854, baseline=1.51.1-SNAPSHOT~6fc7077594

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.452 ms) : 0, 1452
crashtracking [candidate] (1.427 ms) : 0, 1427
BytebuddyAgent [baseline] (741.046 ms) : 0, 741046
BytebuddyAgent [candidate] (730.411 ms) : 0, 730411
GlobalTracer [baseline] (244.574 ms) : 0, 244574
GlobalTracer [candidate] (241.985 ms) : 0, 241985
AppSec [baseline] (31.048 ms) : 0, 31048
AppSec [candidate] (30.389 ms) : 0, 30389
Debugger [baseline] (6.07 ms) : 0, 6070
Debugger [candidate] (6.013 ms) : 0, 6013
Remote Config [baseline] (665.964 µs) : 0, 666
Remote Config [candidate] (660.126 µs) : 0, 660
Telemetry [baseline] (12.309 ms) : 0, 12309
Telemetry [candidate] (8.269 ms) : 0, 8269
section iast
crashtracking [baseline] (1.431 ms) : 0, 1431
crashtracking [candidate] (1.48 ms) : 0, 1480
BytebuddyAgent [baseline] (852.811 ms) : 0, 852811
BytebuddyAgent [candidate] (859.603 ms) : 0, 859603
GlobalTracer [baseline] (232.109 ms) : 0, 232109
GlobalTracer [candidate] (232.175 ms) : 0, 232175
AppSec [baseline] (26.472 ms) : 0, 26472
AppSec [candidate] (29.121 ms) : 0, 29121
Debugger [baseline] (6.642 ms) : 0, 6642
Debugger [candidate] (7.477 ms) : 0, 7477
Remote Config [baseline] (590.483 µs) : 0, 590
Remote Config [candidate] (595.88 µs) : 0, 596
Telemetry [baseline] (8.045 ms) : 0, 8045
Telemetry [candidate] (7.973 ms) : 0, 7973
IAST [baseline] (29.58 ms) : 0, 29580
IAST [candidate] (27.692 ms) : 0, 27692
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.51.1-SNAPSHOT~c7eeac8854, baseline=1.51.1-SNAPSHOT~6fc7077594

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.044 s) : 0, 1043844
Total [baseline] (10.782 s) : 0, 10781701
Agent [candidate] (1.038 s) : 0, 1037593
Total [candidate] (10.718 s) : 0, 10717679
section appsec
Agent [baseline] (1.22 s) : 0, 1220272
Total [baseline] (10.895 s) : 0, 10894542
Agent [candidate] (1.225 s) : 0, 1224962
Total [candidate] (10.929 s) : 0, 10928735
section iast
Agent [baseline] (1.173 s) : 0, 1173384
Total [baseline] (10.97 s) : 0, 10970253
Agent [candidate] (1.178 s) : 0, 1177781
Total [candidate] (10.988 s) : 0, 10988447
section profiling
Agent [baseline] (1.2 s) : 0, 1199678
Total [baseline] (10.914 s) : 0, 10914070
Agent [candidate] (1.194 s) : 0, 1193870
Total [candidate] (10.914 s) : 0, 10913505
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.044 s -
Agent appsec 1.22 s 176.428 ms (16.9%)
Agent iast 1.173 s 129.54 ms (12.4%)
Agent profiling 1.2 s 155.834 ms (14.9%)
Total tracing 10.782 s -
Total appsec 10.895 s 112.842 ms (1.0%)
Total iast 10.97 s 188.553 ms (1.7%)
Total profiling 10.914 s 132.37 ms (1.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.038 s -
Agent appsec 1.225 s 187.369 ms (18.1%)
Agent iast 1.178 s 140.188 ms (13.5%)
Agent profiling 1.194 s 156.277 ms (15.1%)
Total tracing 10.718 s -
Total appsec 10.929 s 211.056 ms (2.0%)
Total iast 10.988 s 270.769 ms (2.5%)
Total profiling 10.914 s 195.826 ms (1.8%) 
gantt
    title petclinic - break down per module: candidate=1.51.1-SNAPSHOT~c7eeac8854, baseline=1.51.1-SNAPSHOT~6fc7077594

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.43 ms) : 0, 1430
crashtracking [candidate] (1.431 ms) : 0, 1431
BytebuddyAgent [baseline] (730.048 ms) : 0, 730048
BytebuddyAgent [candidate] (729.081 ms) : 0, 729081
GlobalTracer [baseline] (241.519 ms) : 0, 241519
GlobalTracer [candidate] (240.975 ms) : 0, 240975
AppSec [baseline] (30.516 ms) : 0, 30516
AppSec [candidate] (30.326 ms) : 0, 30326
Debugger [baseline] (5.971 ms) : 0, 5971
Debugger [candidate] (5.999 ms) : 0, 5999
Remote Config [baseline] (649.757 µs) : 0, 650
Remote Config [candidate] (642.583 µs) : 0, 643
Telemetry [baseline] (12.695 ms) : 0, 12695
Telemetry [candidate] (8.161 ms) : 0, 8161
section appsec
crashtracking [baseline] (1.435 ms) : 0, 1435
crashtracking [candidate] (1.448 ms) : 0, 1448
BytebuddyAgent [baseline] (752.475 ms) : 0, 752475
BytebuddyAgent [candidate] (755.954 ms) : 0, 755954
GlobalTracer [baseline] (235.173 ms) : 0, 235173
GlobalTracer [candidate] (235.952 ms) : 0, 235952
IAST [baseline] (23.596 ms) : 0, 23596
IAST [candidate] (23.697 ms) : 0, 23697
AppSec [baseline] (169.897 ms) : 0, 169897
AppSec [candidate] (167.598 ms) : 0, 167598
Debugger [baseline] (7.243 ms) : 0, 7243
Debugger [candidate] (10.167 ms) : 0, 10167
Remote Config [baseline] (631.432 µs) : 0, 631
Remote Config [candidate] (608.434 µs) : 0, 608
Telemetry [baseline] (8.404 ms) : 0, 8404
Telemetry [candidate] (8.157 ms) : 0, 8157
section iast
crashtracking [baseline] (1.427 ms) : 0, 1427
crashtracking [candidate] (1.435 ms) : 0, 1435
BytebuddyAgent [baseline] (846.845 ms) : 0, 846845
BytebuddyAgent [candidate] (850.261 ms) : 0, 850261
GlobalTracer [baseline] (231.419 ms) : 0, 231419
GlobalTracer [candidate] (232.978 ms) : 0, 232978
IAST [baseline] (29.123 ms) : 0, 29123
IAST [candidate] (28.331 ms) : 0, 28331
AppSec [baseline] (26.462 ms) : 0, 26462
AppSec [candidate] (28.493 ms) : 0, 28493
Debugger [baseline] (8.33 ms) : 0, 8330
Debugger [candidate] (6.566 ms) : 0, 6566
Remote Config [baseline] (590.663 µs) : 0, 591
Remote Config [candidate] (607.17 µs) : 0, 607
Telemetry [baseline] (8.114 ms) : 0, 8114
Telemetry [candidate] (8.009 ms) : 0, 8009
section profiling
crashtracking [baseline] (1.414 ms) : 0, 1414
crashtracking [candidate] (1.407 ms) : 0, 1407
BytebuddyAgent [baseline] (765.964 ms) : 0, 765964
BytebuddyAgent [candidate] (765.22 ms) : 0, 765220
GlobalTracer [baseline] (221.685 ms) : 0, 221685
GlobalTracer [candidate] (221.174 ms) : 0, 221174
AppSec [baseline] (30.729 ms) : 0, 30729
AppSec [candidate] (30.21 ms) : 0, 30210
Debugger [baseline] (6.253 ms) : 0, 6253
Debugger [candidate] (6.294 ms) : 0, 6294
Remote Config [baseline] (710.082 µs) : 0, 710
Remote Config [candidate] (667.841 µs) : 0, 668
Telemetry [baseline] (15.316 ms) : 0, 15316
Telemetry [candidate] (11.803 ms) : 0, 11803
ProfilingAgent [baseline] (108.837 ms) : 0, 108837
ProfilingAgent [candidate] (108.326 ms) : 0, 108326
Profiling [baseline] (108.86 ms) : 0, 108860
Profiling [candidate] (108.349 ms) : 0, 108349
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master daniel.mohedano/failed-test-replay
git_commit_date 1753268392 1753280286
git_commit_sha 6fc7077 c7eeac8
release_version 1.51.1-SNAPSHOT~6fc7077594 1.51.1-SNAPSHOT~c7eeac8854
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1753281943 1753281943
ci_job_id 1043474969 1043474969
ci_pipeline_id 71456628 71456628
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-vizbue04 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-vizbue04 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 4 performance regressions! Performance is the same for 7 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:tracing:high_load worse
[+239.994µs; +469.012µs] or [+3.328%; +6.504%]		 unstable
[-112.345op/s; +52.533op/s] or [-17.504%; +8.185%]		 7.565ms 611.906op/s 7.211ms 641.812op/s
scenario:load:insecure-bank:iast:high_load worse
[+223.621µs; +556.279µs] or [+2.417%; +6.012%]		 unstable
[-82.244op/s; +41.744op/s] or [-16.391%; +8.320%]		 9.642ms 481.500op/s 9.252ms 501.750op/s
scenario:load:petclinic:code_origins:high_load worse
[+1.801ms; +2.641ms] or [+4.088%; +5.996%]		 unstable
[-12.814op/s; +2.689op/s] or [-12.069%; +2.532%]		 46.268ms 101.112op/s 44.047ms 106.175op/s
scenario:load:petclinic:profiling:high_load better
[-3.238ms; -2.253ms] or [-6.404%; -4.455%]		 unstable
[-1.998op/s; +12.648op/s] or [-2.159%; +13.666%]		 47.820ms 97.875op/s 50.566ms 92.550op/s
scenario:load:petclinic:appsec:high_load worse
[+1.594ms; +2.518ms] or [+3.398%; +5.369%]		 unstable
[-11.294op/s; +3.119op/s] or [-11.322%; +3.127%]		 48.961ms 95.662op/s 46.905ms 99.750op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~c7eeac8854, baseline=1.51.1-SNAPSHOT~6fc7077594
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.518 ms) : 4465, 4571
.   : milestone, 4518,
iast (9.252 ms) : 9102, 9403
.   : milestone, 9252,
iast_FULL (14.326 ms) : 14040, 14612
.   : milestone, 14326,
iast_GLOBAL (10.586 ms) : 10396, 10775
.   : milestone, 10586,
profiling (8.685 ms) : 8554, 8816
.   : milestone, 8685,
tracing (7.211 ms) : 7104, 7317
.   : milestone, 7211,
section candidate
no_agent (4.432 ms) : 4382, 4483
.   : milestone, 4432,
iast (9.642 ms) : 9484, 9801
.   : milestone, 9642,
iast_FULL (14.144 ms) : 13862, 14425
.   : milestone, 14144,
iast_GLOBAL (10.274 ms) : 10092, 10456
.   : milestone, 10274,
profiling (8.497 ms) : 8370, 8624
.   : milestone, 8497,
tracing (7.565 ms) : 7459, 7672
.   : milestone, 7565,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.518 ms [4.465 ms, 4.571 ms] -
iast 9.252 ms [9.102 ms, 9.403 ms] 4.734 ms (104.8%)
iast_FULL 14.326 ms [14.04 ms, 14.612 ms] 9.808 ms (217.1%)
iast_GLOBAL 10.586 ms [10.396 ms, 10.775 ms] 6.068 ms (134.3%)
profiling 8.685 ms [8.554 ms, 8.816 ms] 4.167 ms (92.2%)
tracing 7.211 ms [7.104 ms, 7.317 ms] 2.693 ms (59.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.432 ms [4.382 ms, 4.483 ms] -
iast 9.642 ms [9.484 ms, 9.801 ms] 5.21 ms (117.5%)
iast_FULL 14.144 ms [13.862 ms, 14.425 ms] 9.711 ms (219.1%)
iast_GLOBAL 10.274 ms [10.092 ms, 10.456 ms] 5.842 ms (131.8%)
profiling 8.497 ms [8.37 ms, 8.624 ms] 4.065 ms (91.7%)
tracing 7.565 ms [7.459 ms, 7.672 ms] 3.133 ms (70.7%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~c7eeac8854, baseline=1.51.1-SNAPSHOT~6fc7077594
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.227 ms) : 35931, 36524
.   : milestone, 36227,
appsec (46.905 ms) : 46490, 47319
.   : milestone, 46905,
code_origins (44.047 ms) : 43665, 44429
.   : milestone, 44047,
iast (44.21 ms) : 43826, 44593
.   : milestone, 44210,
profiling (50.566 ms) : 50093, 51039
.   : milestone, 50566,
tracing (44.753 ms) : 44364, 45142
.   : milestone, 44753,
section candidate
no_agent (35.524 ms) : 35236, 35811
.   : milestone, 35524,
appsec (48.961 ms) : 48516, 49405
.   : milestone, 48961,
code_origins (46.268 ms) : 45869, 46666
.   : milestone, 46268,
iast (44.721 ms) : 44309, 45133
.   : milestone, 44721,
profiling (47.82 ms) : 47378, 48263
.   : milestone, 47820,
tracing (45.49 ms) : 45098, 45883
.   : milestone, 45490,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.227 ms [35.931 ms, 36.524 ms] -
appsec 46.905 ms [46.49 ms, 47.319 ms] 10.677 ms (29.5%)
code_origins 44.047 ms [43.665 ms, 44.429 ms] 7.819 ms (21.6%)
iast 44.21 ms [43.826 ms, 44.593 ms] 7.982 ms (22.0%)
profiling 50.566 ms [50.093 ms, 51.039 ms] 14.339 ms (39.6%)
tracing 44.753 ms [44.364 ms, 45.142 ms] 8.525 ms (23.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 35.524 ms [35.236 ms, 35.811 ms] -
appsec 48.961 ms [48.516 ms, 49.405 ms] 13.437 ms (37.8%)
code_origins 46.268 ms [45.869 ms, 46.666 ms] 10.744 ms (30.2%)
iast 44.721 ms [44.309 ms, 45.133 ms] 9.197 ms (25.9%)
profiling 47.82 ms [47.378 ms, 48.263 ms] 12.297 ms (34.6%)
tracing 45.49 ms [45.098 ms, 45.883 ms] 9.966 ms (28.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master daniel.mohedano/failed-test-replay
git_commit_date 1753268392 1753280286
git_commit_sha 6fc7077 c7eeac8
release_version 1.51.1-SNAPSHOT~6fc7077594 1.51.1-SNAPSHOT~c7eeac8854
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1753282430 1753282430
ci_job_id 1043474970 1043474970
ci_pipeline_id 71456628 71456628
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-uyb30puh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-uyb30puh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 1 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-1.395ms; -1.055ms] or [-37.968%; -28.716%]		 2.450ms 3.675ms
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~c7eeac8854, baseline=1.51.1-SNAPSHOT~6fc7077594
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (3.675 ms) : 3458, 3892
.   : milestone, 3675,
iast (2.198 ms) : 2135, 2261
.   : milestone, 2198,
iast_GLOBAL (2.244 ms) : 2181, 2307
.   : milestone, 2244,
profiling (2.038 ms) : 1988, 2089
.   : milestone, 2038,
tracing (2.015 ms) : 1966, 2063
.   : milestone, 2015,
section candidate
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (2.45 ms) : 2397, 2502
.   : milestone, 2450,
iast (2.195 ms) : 2132, 2258
.   : milestone, 2195,
iast_GLOBAL (2.24 ms) : 2176, 2303
.   : milestone, 2240,
profiling (2.487 ms) : 2315, 2660
.   : milestone, 2487,
tracing (2.008 ms) : 1960, 2057
.   : milestone, 2008,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.46 ms, 1.483 ms] -
appsec 3.675 ms [3.458 ms, 3.892 ms] 2.203 ms (149.7%)
iast 2.198 ms [2.135 ms, 2.261 ms] 726.14 µs (49.3%)
iast_GLOBAL 2.244 ms [2.181 ms, 2.307 ms] 772.438 µs (52.5%)
profiling 2.038 ms [1.988 ms, 2.089 ms] 566.377 µs (38.5%)
tracing 2.015 ms [1.966 ms, 2.063 ms] 543.181 µs (36.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.46 ms, 1.483 ms] -
appsec 2.45 ms [2.397 ms, 2.502 ms] 977.972 µs (66.5%)
iast 2.195 ms [2.132 ms, 2.258 ms] 723.604 µs (49.2%)
iast_GLOBAL 2.24 ms [2.176 ms, 2.303 ms] 768.055 µs (52.2%)
profiling 2.487 ms [2.315 ms, 2.66 ms] 1.016 ms (69.0%)
tracing 2.008 ms [1.96 ms, 2.057 ms] 536.83 µs (36.5%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~c7eeac8854, baseline=1.51.1-SNAPSHOT~6fc7077594
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.274 s) : 15274000, 15274000
.   : milestone, 15274000,
appsec (14.734 s) : 14734000, 14734000
.   : milestone, 14734000,
iast (18.44 s) : 18440000, 18440000
.   : milestone, 18440000,
iast_GLOBAL (17.969 s) : 17969000, 17969000
.   : milestone, 17969000,
profiling (15.016 s) : 15016000, 15016000
.   : milestone, 15016000,
tracing (15.12 s) : 15120000, 15120000
.   : milestone, 15120000,
section candidate
no_agent (15.031 s) : 15031000, 15031000
.   : milestone, 15031000,
appsec (14.857 s) : 14857000, 14857000
.   : milestone, 14857000,
iast (18.749 s) : 18749000, 18749000
.   : milestone, 18749000,
iast_GLOBAL (17.909 s) : 17909000, 17909000
.   : milestone, 17909000,
profiling (15.544 s) : 15544000, 15544000
.   : milestone, 15544000,
tracing (14.763 s) : 14763000, 14763000
.   : milestone, 14763000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.274 s [15.274 s, 15.274 s] -
appsec 14.734 s [14.734 s, 14.734 s] -540.0 ms (-3.5%)
iast 18.44 s [18.44 s, 18.44 s] 3.166 s (20.7%)
iast_GLOBAL 17.969 s [17.969 s, 17.969 s] 2.695 s (17.6%)
profiling 15.016 s [15.016 s, 15.016 s] -258.0 ms (-1.7%)
tracing 15.12 s [15.12 s, 15.12 s] -154.0 ms (-1.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.031 s [15.031 s, 15.031 s] -
appsec 14.857 s [14.857 s, 14.857 s] -174.0 ms (-1.2%)
iast 18.749 s [18.749 s, 18.749 s] 3.718 s (24.7%)
iast_GLOBAL 17.909 s [17.909 s, 17.909 s] 2.878 s (19.1%)
profiling 15.544 s [15.544 s, 15.544 s] 513.0 ms (3.4%)
tracing 14.763 s [14.763 s, 14.763 s] -268.0 ms (-1.8%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility comp: debugger Dynamic Instrumentation tag: do not merge Do not merge changes type: enhancement Enhancements and improvements
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@daniel-mohedano