Move macrobenchmarks to `apm-sdks-benchmarks` repo #10066

sarahchen6 · 2025-12-02T17:27:24Z

WIP!!

What Does This Do

Test the apm-sdks-benchmarks repo implementation of macrobenchmarks: https://github.com/DataDog/apm-sdks-benchmarks/pull/34

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-12-02T18:47:56Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-macrobenchmarks
git_commit_date	1765222535	1765227109
git_commit_sha	`94e770c`	`4485a05`
release_version	1.57.0-SNAPSHOT~94e770cd67	1.57.0-SNAPSHOT~4485a059e5

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1765229013	1765229013
ci_job_id	1277420892	1277420892
ci_pipeline_id	85406736	85406736
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-3-62gymulx 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-3-62gymulx 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 56 metrics, 9 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~4485a059e5, baseline=1.57.0-SNAPSHOT~94e770cd67

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.084 s) : 0, 1084340
Total [baseline] (8.739 s) : 0, 8739381
Agent [candidate] (1.081 s) : 0, 1080832
Total [candidate] (8.787 s) : 0, 8787266
section iast
Agent [baseline] (1.221 s) : 0, 1220984
Total [baseline] (9.453 s) : 0, 9453101
Agent [candidate] (1.228 s) : 0, 1227520
Total [candidate] (9.515 s) : 0, 9515340

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.084 s	-
Agent	iast	1.221 s	136.643 ms (12.6%)
Total	tracing	8.739 s	-
Total	iast	9.453 s	713.72 ms (8.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.081 s	-
Agent	iast	1.228 s	146.688 ms (13.6%)
Total	tracing	8.787 s	-
Total	iast	9.515 s	728.074 ms (8.3%)

gantt
    title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~4485a059e5, baseline=1.57.0-SNAPSHOT~94e770cd67

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.203 ms) : 0, 1203
crashtracking [candidate] (1.207 ms) : 0, 1207
BytebuddyAgent [baseline] (651.355 ms) : 0, 651355
BytebuddyAgent [candidate] (649.279 ms) : 0, 649279
GlobalTracer [baseline] (282.897 ms) : 0, 282897
GlobalTracer [candidate] (282.26 ms) : 0, 282260
AppSec [baseline] (32.558 ms) : 0, 32558
AppSec [candidate] (32.263 ms) : 0, 32263
Debugger [baseline] (67.243 ms) : 0, 67243
Debugger [candidate] (67.03 ms) : 0, 67030
Remote Config [baseline] (668.192 µs) : 0, 668
Remote Config [candidate] (632.702 µs) : 0, 633
Telemetry [baseline] (9.048 ms) : 0, 9048
Telemetry [candidate] (8.899 ms) : 0, 8899
Flare Poller [baseline] (3.753 ms) : 0, 3753
Flare Poller [candidate] (3.717 ms) : 0, 3717
section iast
crashtracking [baseline] (1.197 ms) : 0, 1197
crashtracking [candidate] (1.209 ms) : 0, 1209
BytebuddyAgent [baseline] (789.01 ms) : 0, 789010
BytebuddyAgent [candidate] (795.077 ms) : 0, 795077
GlobalTracer [baseline] (255.363 ms) : 0, 255363
GlobalTracer [candidate] (256.492 ms) : 0, 256492
AppSec [baseline] (35.711 ms) : 0, 35711
AppSec [candidate] (33.507 ms) : 0, 33507
Debugger [baseline] (64.806 ms) : 0, 64806
Debugger [candidate] (65.558 ms) : 0, 65558
Remote Config [baseline] (560.19 µs) : 0, 560
Remote Config [candidate] (562.457 µs) : 0, 562
Telemetry [baseline] (8.387 ms) : 0, 8387
Telemetry [candidate] (8.416 ms) : 0, 8416
Flare Poller [baseline] (3.434 ms) : 0, 3434
Flare Poller [candidate] (3.442 ms) : 0, 3442
IAST [baseline] (27.076 ms) : 0, 27076
IAST [candidate] (27.734 ms) : 0, 27734

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~4485a059e5, baseline=1.57.0-SNAPSHOT~94e770cd67

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.088 s) : 0, 1087756
Total [baseline] (10.856 s) : 0, 10856202
Agent [candidate] (1.081 s) : 0, 1080803
Total [candidate] (10.795 s) : 0, 10794786
section appsec
Agent [baseline] (1.264 s) : 0, 1264109
Total [baseline] (11.185 s) : 0, 11184586
Agent [candidate] (1.282 s) : 0, 1282360
Total [candidate] (11.334 s) : 0, 11334342
section iast
Agent [baseline] (1.232 s) : 0, 1231548
Total [baseline] (11.202 s) : 0, 11202148
Agent [candidate] (1.223 s) : 0, 1222590
Total [candidate] (11.146 s) : 0, 11146332
section profiling
Agent [baseline] (1.205 s) : 0, 1205285
Total [baseline] (11.049 s) : 0, 11049099
Agent [candidate] (1.211 s) : 0, 1211434
Total [candidate] (11.025 s) : 0, 11024710

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.088 s	-
Agent	appsec	1.264 s	176.353 ms (16.2%)
Agent	iast	1.232 s	143.793 ms (13.2%)
Agent	profiling	1.205 s	117.529 ms (10.8%)
Total	tracing	10.856 s	-
Total	appsec	11.185 s	328.384 ms (3.0%)
Total	iast	11.202 s	345.946 ms (3.2%)
Total	profiling	11.049 s	192.896 ms (1.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.081 s	-
Agent	appsec	1.282 s	201.557 ms (18.6%)
Agent	iast	1.223 s	141.786 ms (13.1%)
Agent	profiling	1.211 s	130.631 ms (12.1%)
Total	tracing	10.795 s	-
Total	appsec	11.334 s	539.556 ms (5.0%)
Total	iast	11.146 s	351.546 ms (3.3%)
Total	profiling	11.025 s	229.924 ms (2.1%)

gantt
    title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~4485a059e5, baseline=1.57.0-SNAPSHOT~94e770cd67

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.215 ms) : 0, 1215
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (652.774 ms) : 0, 652774
BytebuddyAgent [candidate] (647.459 ms) : 0, 647459
GlobalTracer [baseline] (283.439 ms) : 0, 283439
GlobalTracer [candidate] (281.96 ms) : 0, 281960
AppSec [baseline] (32.628 ms) : 0, 32628
AppSec [candidate] (32.493 ms) : 0, 32493
Debugger [baseline] (68.493 ms) : 0, 68493
Debugger [candidate] (68.728 ms) : 0, 68728
Remote Config [baseline] (673.253 µs) : 0, 673
Remote Config [candidate] (652.908 µs) : 0, 653
Telemetry [baseline] (9.094 ms) : 0, 9094
Telemetry [candidate] (9.077 ms) : 0, 9077
Flare Poller [baseline] (3.791 ms) : 0, 3791
Flare Poller [candidate] (3.816 ms) : 0, 3816
section appsec
crashtracking [baseline] (1.188 ms) : 0, 1188
crashtracking [candidate] (1.218 ms) : 0, 1218
BytebuddyAgent [baseline] (688.442 ms) : 0, 688442
BytebuddyAgent [candidate] (700.006 ms) : 0, 700006
GlobalTracer [baseline] (259.711 ms) : 0, 259711
GlobalTracer [candidate] (263.502 ms) : 0, 263502
AppSec [baseline] (174.888 ms) : 0, 174888
AppSec [candidate] (176.136 ms) : 0, 176136
Debugger [baseline] (65.948 ms) : 0, 65948
Debugger [candidate] (66.8 ms) : 0, 66800
Remote Config [baseline] (755.867 µs) : 0, 756
Remote Config [candidate] (726.957 µs) : 0, 727
Telemetry [baseline] (9.097 ms) : 0, 9097
Telemetry [candidate] (9.221 ms) : 0, 9221
Flare Poller [baseline] (3.91 ms) : 0, 3910
Flare Poller [candidate] (4.0 ms) : 0, 4000
IAST [baseline] (24.647 ms) : 0, 24647
IAST [candidate] (24.994 ms) : 0, 24994
section iast
crashtracking [baseline] (1.218 ms) : 0, 1218
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (795.773 ms) : 0, 795773
BytebuddyAgent [candidate] (789.515 ms) : 0, 789515
GlobalTracer [baseline] (257.118 ms) : 0, 257118
GlobalTracer [candidate] (255.673 ms) : 0, 255673
AppSec [baseline] (34.962 ms) : 0, 34962
AppSec [candidate] (33.538 ms) : 0, 33538
Debugger [baseline] (67.217 ms) : 0, 67217
Debugger [candidate] (67.799 ms) : 0, 67799
Remote Config [baseline] (592.359 µs) : 0, 592
Remote Config [candidate] (576.952 µs) : 0, 577
Telemetry [baseline] (8.399 ms) : 0, 8399
Telemetry [candidate] (8.437 ms) : 0, 8437
Flare Poller [baseline] (3.446 ms) : 0, 3446
Flare Poller [candidate] (3.471 ms) : 0, 3471
IAST [baseline] (27.24 ms) : 0, 27240
IAST [candidate] (27.026 ms) : 0, 27026
section profiling
crashtracking [baseline] (1.201 ms) : 0, 1201
crashtracking [candidate] (1.193 ms) : 0, 1193
BytebuddyAgent [baseline] (702.348 ms) : 0, 702348
BytebuddyAgent [candidate] (706.841 ms) : 0, 706841
GlobalTracer [baseline] (221.244 ms) : 0, 221244
GlobalTracer [candidate] (222.068 ms) : 0, 222068
AppSec [baseline] (32.15 ms) : 0, 32150
AppSec [candidate] (32.15 ms) : 0, 32150
Debugger [baseline] (68.001 ms) : 0, 68001
Debugger [candidate] (68.046 ms) : 0, 68046
Remote Config [baseline] (611.868 µs) : 0, 612
Remote Config [candidate] (628.457 µs) : 0, 628
Telemetry [baseline] (8.923 ms) : 0, 8923
Telemetry [candidate] (8.927 ms) : 0, 8927
Flare Poller [baseline] (3.743 ms) : 0, 3743
Flare Poller [candidate] (3.777 ms) : 0, 3777
ProfilingAgent [baseline] (97.508 ms) : 0, 97508
ProfilingAgent [candidate] (97.898 ms) : 0, 97898
Profiling [baseline] (98.073 ms) : 0, 98073
Profiling [candidate] (98.469 ms) : 0, 98469

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-macrobenchmarks
git_commit_date	1765222535	1765227109
git_commit_sha	`94e770c`	`4485a05`
release_version	1.57.0-SNAPSHOT~94e770cd67	1.57.0-SNAPSHOT~4485a059e5

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1765229522	1765229522
ci_job_id	1277420893	1277420893
ci_pipeline_id	85406736	85406736
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-2-o1499gfh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-2-o1499gfh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 4 performance regressions! Performance is the same for 13 metrics, 17 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load	worse [+80.880µs; +192.588µs] or [+2.990%; +7.121%]	worse [+204.214µs; +769.624µs] or [+2.692%; +10.145%]	unstable [-212.265op/s; +80.453op/s] or [-15.872%; +6.016%]	2.841ms	8.073ms	1271.438op/s	2.705ms	7.586ms	1337.344op/s
scenario:load:insecure-bank:iast_FULL:high_load	same [-209.016µs; +138.770µs] or [-3.961%; +2.630%]	better [-1055.515µs; -282.195µs] or [-8.170%; -2.184%]	unstable [-62.086op/s; +99.774op/s] or [-8.077%; +12.980%]	5.242ms	12.251ms	787.500op/s	5.277ms	12.920ms	768.656op/s
scenario:load:petclinic:appsec:high_load	worse [+382.452µs; +854.784µs] or [+2.112%; +4.721%]	same [-116.680µs; +1233.912µs] or [-0.395%; +4.178%]	unstable [-33.000op/s; +18.437op/s] or [-13.069%; +7.302%]	18.726ms	30.092ms	245.219op/s	18.107ms	29.534ms	252.500op/s
scenario:load:petclinic:profiling:high_load	worse [+0.903ms; +1.915ms] or [+4.881%; +10.347%]	unsure [+0.390ms; +2.438ms] or [+1.292%; +8.080%]	unstable [-39.913op/s; +10.850op/s] or [-16.126%; +4.384%]	19.916ms	31.585ms	232.969op/s	18.507ms	30.171ms	247.500op/s
scenario:load:petclinic:code_origins:high_load	better [-1382.969µs; -466.246µs] or [-7.516%; -2.534%]	unsure [-2.204ms; -0.499ms] or [-7.380%; -1.671%]	unstable [-14.974op/s; +39.599op/s] or [-6.015%; +15.907%]	17.477ms	28.509ms	261.250op/s	18.401ms	29.860ms	248.938op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~4485a059e5, baseline=1.57.0-SNAPSHOT~94e770cd67
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.227 ms) : 1214, 1239
.   : milestone, 1227,
iast (3.241 ms) : 3204, 3279
.   : milestone, 3241,
iast_FULL (6.018 ms) : 5957, 6080
.   : milestone, 6018,
iast_GLOBAL (3.425 ms) : 3375, 3475
.   : milestone, 3425,
profiling (2.061 ms) : 2043, 2080
.   : milestone, 2061,
tracing (1.81 ms) : 1796, 1825
.   : milestone, 1810,
section candidate
no_agent (1.205 ms) : 1193, 1216
.   : milestone, 1205,
iast (3.305 ms) : 3260, 3350
.   : milestone, 3305,
iast_FULL (5.872 ms) : 5813, 5930
.   : milestone, 5872,
iast_GLOBAL (3.607 ms) : 3550, 3664
.   : milestone, 3607,
profiling (1.997 ms) : 1979, 2016
.   : milestone, 1997,
tracing (1.814 ms) : 1798, 1829
.   : milestone, 1814,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.227 ms [1.214 ms, 1.239 ms]	-
iast	3.241 ms [3.204 ms, 3.279 ms]	2.015 ms (164.3%)
iast_FULL	6.018 ms [5.957 ms, 6.08 ms]	4.792 ms (390.6%)
iast_GLOBAL	3.425 ms [3.375 ms, 3.475 ms]	2.198 ms (179.2%)
profiling	2.061 ms [2.043 ms, 2.08 ms]	834.698 µs (68.0%)
tracing	1.81 ms [1.796 ms, 1.825 ms]	583.569 µs (47.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.205 ms [1.193 ms, 1.216 ms]	-
iast	3.305 ms [3.26 ms, 3.35 ms]	2.1 ms (174.4%)
iast_FULL	5.872 ms [5.813 ms, 5.93 ms]	4.667 ms (387.4%)
iast_GLOBAL	3.607 ms [3.55 ms, 3.664 ms]	2.402 ms (199.4%)
profiling	1.997 ms [1.979 ms, 2.016 ms]	792.637 µs (65.8%)
tracing	1.814 ms [1.798 ms, 1.829 ms]	608.992 µs (50.6%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~4485a059e5, baseline=1.57.0-SNAPSHOT~94e770cd67
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.157 ms) : 17970, 18345
.   : milestone, 18157,
appsec (18.485 ms) : 18300, 18670
.   : milestone, 18485,
code_origins (18.744 ms) : 18552, 18936
.   : milestone, 18744,
iast (17.833 ms) : 17658, 18009
.   : milestone, 17833,
profiling (18.859 ms) : 18667, 19052
.   : milestone, 18859,
tracing (18.097 ms) : 17916, 18277
.   : milestone, 18097,
section candidate
no_agent (18.141 ms) : 17960, 18321
.   : milestone, 18141,
appsec (19.033 ms) : 18841, 19225
.   : milestone, 19033,
code_origins (17.862 ms) : 17688, 18037
.   : milestone, 17862,
iast (17.745 ms) : 17570, 17919
.   : milestone, 17745,
profiling (20.04 ms) : 19837, 20243
.   : milestone, 20040,
tracing (17.888 ms) : 17712, 18064
.   : milestone, 17888,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.157 ms [17.97 ms, 18.345 ms]	-
appsec	18.485 ms [18.3 ms, 18.67 ms]	327.259 µs (1.8%)
code_origins	18.744 ms [18.552 ms, 18.936 ms]	586.563 µs (3.2%)
iast	17.833 ms [17.658 ms, 18.009 ms]	-324.157 µs (-1.8%)
profiling	18.859 ms [18.667 ms, 19.052 ms]	701.842 µs (3.9%)
tracing	18.097 ms [17.916 ms, 18.277 ms]	-60.893 µs (-0.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.141 ms [17.96 ms, 18.321 ms]	-
appsec	19.033 ms [18.841 ms, 19.225 ms]	892.422 µs (4.9%)
code_origins	17.862 ms [17.688 ms, 18.037 ms]	-278.654 µs (-1.5%)
iast	17.745 ms [17.57 ms, 17.919 ms]	-396.234 µs (-2.2%)
profiling	20.04 ms [19.837 ms, 20.243 ms]	1.899 ms (10.5%)
tracing	17.888 ms [17.712 ms, 18.064 ms]	-252.566 µs (-1.4%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-macrobenchmarks
git_commit_date	1765222535	1765227109
git_commit_sha	`94e770c`	`4485a05`
release_version	1.57.0-SNAPSHOT~94e770cd67	1.57.0-SNAPSHOT~4485a059e5

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1765229161	1765229161
ci_job_id	1277420894	1277420894
ci_pipeline_id	85406736	85406736
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-4-1hftx8gl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-4-1hftx8gl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~4485a059e5, baseline=1.57.0-SNAPSHOT~94e770cd67
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.483 ms) : 1472, 1495
.   : milestone, 1483,
appsec (3.73 ms) : 3509, 3951
.   : milestone, 3730,
iast (2.229 ms) : 2164, 2294
.   : milestone, 2229,
iast_GLOBAL (2.275 ms) : 2210, 2341
.   : milestone, 2275,
profiling (2.101 ms) : 2046, 2155
.   : milestone, 2101,
tracing (2.075 ms) : 2024, 2126
.   : milestone, 2075,
section candidate
no_agent (1.481 ms) : 1470, 1493
.   : milestone, 1481,
appsec (3.771 ms) : 3549, 3993
.   : milestone, 3771,
iast (2.229 ms) : 2164, 2293
.   : milestone, 2229,
iast_GLOBAL (2.275 ms) : 2210, 2341
.   : milestone, 2275,
profiling (2.087 ms) : 2034, 2140
.   : milestone, 2087,
tracing (2.063 ms) : 2011, 2114
.   : milestone, 2063,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.483 ms [1.472 ms, 1.495 ms]	-
appsec	3.73 ms [3.509 ms, 3.951 ms]	2.247 ms (151.5%)
iast	2.229 ms [2.164 ms, 2.294 ms]	746.012 µs (50.3%)
iast_GLOBAL	2.275 ms [2.21 ms, 2.341 ms]	792.406 µs (53.4%)
profiling	2.101 ms [2.046 ms, 2.155 ms]	617.816 µs (41.7%)
tracing	2.075 ms [2.024 ms, 2.126 ms]	591.847 µs (39.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.481 ms [1.47 ms, 1.493 ms]	-
appsec	3.771 ms [3.549 ms, 3.993 ms]	2.29 ms (154.6%)
iast	2.229 ms [2.164 ms, 2.293 ms]	747.42 µs (50.5%)
iast_GLOBAL	2.275 ms [2.21 ms, 2.341 ms]	793.978 µs (53.6%)
profiling	2.087 ms [2.034 ms, 2.14 ms]	605.884 µs (40.9%)
tracing	2.063 ms [2.011 ms, 2.114 ms]	581.247 µs (39.2%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~4485a059e5, baseline=1.57.0-SNAPSHOT~94e770cd67
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.015 s) : 15015000, 15015000
.   : milestone, 15015000,
appsec (14.535 s) : 14535000, 14535000
.   : milestone, 14535000,
iast (17.971 s) : 17971000, 17971000
.   : milestone, 17971000,
iast_GLOBAL (18.159 s) : 18159000, 18159000
.   : milestone, 18159000,
profiling (14.555 s) : 14555000, 14555000
.   : milestone, 14555000,
tracing (14.688 s) : 14688000, 14688000
.   : milestone, 14688000,
section candidate
no_agent (15.505 s) : 15505000, 15505000
.   : milestone, 15505000,
appsec (14.623 s) : 14623000, 14623000
.   : milestone, 14623000,
iast (18.579 s) : 18579000, 18579000
.   : milestone, 18579000,
iast_GLOBAL (17.877 s) : 17877000, 17877000
.   : milestone, 17877000,
profiling (15.236 s) : 15236000, 15236000
.   : milestone, 15236000,
tracing (14.914 s) : 14914000, 14914000
.   : milestone, 14914000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.015 s [15.015 s, 15.015 s]	-
appsec	14.535 s [14.535 s, 14.535 s]	-480.0 ms (-3.2%)
iast	17.971 s [17.971 s, 17.971 s]	2.956 s (19.7%)
iast_GLOBAL	18.159 s [18.159 s, 18.159 s]	3.144 s (20.9%)
profiling	14.555 s [14.555 s, 14.555 s]	-460.0 ms (-3.1%)
tracing	14.688 s [14.688 s, 14.688 s]	-327.0 ms (-2.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.505 s [15.505 s, 15.505 s]	-
appsec	14.623 s [14.623 s, 14.623 s]	-882.0 ms (-5.7%)
iast	18.579 s [18.579 s, 18.579 s]	3.074 s (19.8%)
iast_GLOBAL	17.877 s [17.877 s, 17.877 s]	2.372 s (15.3%)
profiling	15.236 s [15.236 s, 15.236 s]	-269.0 ms (-1.7%)
tracing	14.914 s [14.914 s, 14.914 s]	-591.0 ms (-3.8%)

sarahchen6 force-pushed the sarahchen6/move-macrobenchmarks branch from f391147 to 9602f73 Compare December 2, 2025 17:28

sarahchen6 added 2 commits December 2, 2025 11:43

Use apm-sdks-benchmarks macrobenchmarks

d7ff239

Move to gitlab-ci file

b376785

sarahchen6 force-pushed the sarahchen6/move-macrobenchmarks branch from ba1ae21 to b376785 Compare December 2, 2025 17:44

sarahchen6 added 2 commits December 2, 2025 11:46

Test main

bf66715

Fix includes and stages

ddf61d5

sarahchen6 added 3 commits December 2, 2025 17:31

Delete macrobenchmarks file

d69aeaf

Empty commit to run benchmarks again

a9e13c9

Empty commit to run benchmarks again 2

4a173b8

sarahchen6 temporarily deployed to ci-build December 3, 2025 17:59 — with GitHub Actions Inactive

Empty commit to run benchmarks again 3

de5ab06

sarahchen6 temporarily deployed to ci-build December 4, 2025 14:33 — with GitHub Actions Inactive

Empty commit to run benchmarks again 4

6c41b8c

sarahchen6 temporarily deployed to ci-build December 4, 2025 14:48 — with GitHub Actions Inactive

Merge branch 'master' into sarahchen6/move-macrobenchmarks

e1a52bc

sarahchen6 temporarily deployed to ci-build December 5, 2025 15:38 — with GitHub Actions Inactive

Exclude startup benchmarks from slo check for now

bd827c3

sarahchen6 temporarily deployed to ci-build December 5, 2025 16:24 — with GitHub Actions Inactive

Merge branch 'master' into sarahchen6/move-macrobenchmarks

39bf2ea

sarahchen6 temporarily deployed to ci-build December 5, 2025 19:54 — with GitHub Actions Inactive

Merge branch 'master' into sarahchen6/move-macrobenchmarks

4485a05

sarahchen6 temporarily deployed to ci-build December 8, 2025 20:51 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Move macrobenchmarks to `apm-sdks-benchmarks` repo #10066

Move macrobenchmarks to `apm-sdks-benchmarks` repo #10066

Uh oh!

sarahchen6 commented Dec 2, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Dec 2, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Move macrobenchmarks to apm-sdks-benchmarks repo #10066

Are you sure you want to change the base?

Move macrobenchmarks to apm-sdks-benchmarks repo #10066

Uh oh!

Conversation

sarahchen6 commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Move macrobenchmarks to `apm-sdks-benchmarks` repo #10066

Move macrobenchmarks to `apm-sdks-benchmarks` repo #10066

sarahchen6 commented Dec 2, 2025 •

edited

Loading

pr-commenter bot commented Dec 2, 2025 •

edited

Loading