-
Notifications
You must be signed in to change notification settings - Fork 470
ci(iast): update benchmark functions #15563
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 259 ± 3 ms. The average import time from base is: 262 ± 3 ms. The import time difference between this PR and base is: -3.1 ± 0.1 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate avara1986/APPSEC-60151_update_benchmark_2 (9537fe7) with baseline main (256d07a) 📈 Performance Regressions (4 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 18.018µs (SLO: <20.000µs -9.9%) vs baseline: 📈 +21.9% Memory: ✅ 42.743MB (SLO: <43.250MB 🟡 -1.2%) vs baseline: +4.3% ✅ add_inplace_aspectTime: ✅ 14.997µs (SLO: <20.000µs 📉 -25.0%) vs baseline: +0.9% Memory: ✅ 42.979MB (SLO: <43.250MB 🟡 -0.6%) vs baseline: +4.9% ✅ add_inplace_noaspectTime: ✅ 0.339µs (SLO: <10.000µs 📉 -96.6%) vs baseline: +0.4% Memory: ✅ 42.998MB (SLO: <43.000MB 🟡 ~same) vs baseline: +5.1% ✅ add_noaspectTime: ✅ 0.545µs (SLO: <10.000µs 📉 -94.5%) vs baseline: -0.4% Memory: ✅ 42.605MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.6% ✅ bytearray_aspectTime: ✅ 18.101µs (SLO: <30.000µs 📉 -39.7%) vs baseline: +2.1% Memory: ✅ 43.018MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +5.0% ✅ bytearray_extend_aspectTime: ✅ 23.752µs (SLO: <30.000µs 📉 -20.8%) vs baseline: +0.8% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.8% ✅ bytearray_extend_noaspectTime: ✅ 2.667µs (SLO: <10.000µs 📉 -73.3%) vs baseline: -2.7% Memory: ✅ 42.743MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.4% ✅ bytearray_noaspectTime: ✅ 1.457µs (SLO: <10.000µs 📉 -85.4%) vs baseline: +1.5% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.1% ✅ bytes_aspectTime: ✅ 16.627µs (SLO: <20.000µs 📉 -16.9%) vs baseline: +1.5% Memory: ✅ 42.939MB (SLO: <43.000MB 🟡 -0.1%) vs baseline: +4.7% ✅ bytes_noaspectTime: ✅ 1.405µs (SLO: <10.000µs 📉 -86.0%) vs baseline: +0.7% Memory: ✅ 42.743MB (SLO: <43.000MB 🟡 -0.6%) vs baseline: +4.6% ✅ bytesio_aspectTime: ✅ 55.683µs (SLO: <70.000µs 📉 -20.5%) vs baseline: +2.1% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.9% ✅ bytesio_noaspectTime: ✅ 3.263µs (SLO: <10.000µs 📉 -67.4%) vs baseline: +0.9% Memory: ✅ 42.802MB (SLO: <43.500MB 🟡 -1.6%) vs baseline: +4.7% ✅ capitalize_aspectTime: ✅ 14.638µs (SLO: <20.000µs 📉 -26.8%) vs baseline: +0.8% Memory: ✅ 42.979MB (SLO: <43.000MB 🟡 ~same) vs baseline: +5.0% ✅ capitalize_noaspectTime: ✅ 2.558µs (SLO: <10.000µs 📉 -74.4%) vs baseline: ~same Memory: ✅ 42.644MB (SLO: <43.000MB 🟡 -0.8%) vs baseline: +4.3% ✅ casefold_aspectTime: ✅ 14.612µs (SLO: <20.000µs 📉 -26.9%) vs baseline: +0.7% Memory: ✅ 42.979MB (SLO: <43.000MB 🟡 ~same) vs baseline: +4.9% ✅ casefold_noaspectTime: ✅ 3.134µs (SLO: <10.000µs 📉 -68.7%) vs baseline: -0.1% Memory: ✅ 42.625MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.2% ✅ decode_aspectTime: ✅ 15.468µs (SLO: <30.000µs 📉 -48.4%) vs baseline: +1.2% Memory: ✅ 42.841MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +4.6% ✅ decode_noaspectTime: ✅ 1.590µs (SLO: <10.000µs 📉 -84.1%) vs baseline: -0.8% Memory: ✅ 42.782MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.5% ✅ encode_aspectTime: ✅ 17.979µs (SLO: <30.000µs 📉 -40.1%) vs baseline: 📈 +22.5% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.5% ✅ encode_noaspectTime: ✅ 1.473µs (SLO: <10.000µs 📉 -85.3%) vs baseline: -0.2% Memory: ✅ 42.802MB (SLO: <43.000MB 🟡 -0.5%) vs baseline: +5.1% ✅ format_aspectTime: ✅ 169.675µs (SLO: <200.000µs 📉 -15.2%) vs baseline: +0.3% Memory: ✅ 42.920MB (SLO: <43.250MB 🟡 -0.8%) vs baseline: +4.4% ✅ format_map_aspectTime: ✅ 189.858µs (SLO: <200.000µs -5.1%) vs baseline: +0.3% Memory: ✅ 42.998MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.7% ✅ format_map_noaspectTime: ✅ 3.824µs (SLO: <10.000µs 📉 -61.8%) vs baseline: -0.4% Memory: ✅ 42.684MB (SLO: <43.250MB 🟡 -1.3%) vs baseline: +4.4% ✅ format_noaspectTime: ✅ 3.178µs (SLO: <10.000µs 📉 -68.2%) vs baseline: +0.8% Memory: ✅ 42.723MB (SLO: <43.250MB 🟡 -1.2%) vs baseline: +4.3% ✅ index_aspectTime: ✅ 15.354µs (SLO: <20.000µs 📉 -23.2%) vs baseline: +0.9% Memory: ✅ 42.841MB (SLO: <43.250MB 🟡 -0.9%) vs baseline: +4.6% ✅ index_noaspectTime: ✅ 0.462µs (SLO: <10.000µs 📉 -95.4%) vs baseline: +0.8% Memory: ✅ 42.664MB (SLO: <43.000MB 🟡 -0.8%) vs baseline: +4.9% ✅ join_aspectTime: ✅ 17.036µs (SLO: <20.000µs 📉 -14.8%) vs baseline: +1.6% Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.7% ✅ join_noaspectTime: ✅ 1.511µs (SLO: <10.000µs 📉 -84.9%) vs baseline: +2.6% Memory: ✅ 42.684MB (SLO: <43.250MB 🟡 -1.3%) vs baseline: +4.6% ✅ ljust_aspectTime: ✅ 20.854µs (SLO: <30.000µs 📉 -30.5%) vs baseline: +0.7% Memory: ✅ 42.841MB (SLO: <43.250MB 🟡 -0.9%) vs baseline: +4.5% ✅ ljust_noaspectTime: ✅ 2.709µs (SLO: <10.000µs 📉 -72.9%) vs baseline: +2.0% Memory: ✅ 42.585MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +3.9% ✅ lower_aspectTime: ✅ 17.989µs (SLO: <30.000µs 📉 -40.0%) vs baseline: +1.2% Memory: ✅ 42.880MB (SLO: <43.000MB 🟡 -0.3%) vs baseline: +4.5% ✅ lower_noaspectTime: ✅ 2.369µs (SLO: <10.000µs 📉 -76.3%) vs baseline: ~same Memory: ✅ 42.762MB (SLO: <43.250MB 🟡 -1.1%) vs baseline: +4.8% ✅ lstrip_aspectTime: ✅ 17.844µs (SLO: <20.000µs 📉 -10.8%) vs baseline: +1.8% Memory: ✅ 42.900MB (SLO: <43.250MB 🟡 -0.8%) vs baseline: +4.8% ✅ lstrip_noaspectTime: ✅ 1.812µs (SLO: <10.000µs 📉 -81.9%) vs baseline: -1.4% Memory: ✅ 42.762MB (SLO: <43.000MB 🟡 -0.6%) vs baseline: +4.4% ✅ modulo_aspectTime: ✅ 165.374µs (SLO: <200.000µs 📉 -17.3%) vs baseline: +0.6% Memory: ✅ 43.057MB (SLO: <43.500MB 🟡 -1.0%) vs baseline: +4.8% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 178.358µs (SLO: <200.000µs 📉 -10.8%) vs baseline: +3.3% Memory: ✅ 43.116MB (SLO: <43.500MB 🟡 -0.9%) vs baseline: +5.0% ✅ modulo_aspect_for_bytesTime: ✅ 167.294µs (SLO: <200.000µs 📉 -16.4%) vs baseline: +0.4% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.5% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 170.419µs (SLO: <200.000µs 📉 -14.8%) vs baseline: +0.4% Memory: ✅ 43.136MB (SLO: <43.500MB 🟡 -0.8%) vs baseline: +5.0% ✅ modulo_noaspectTime: ✅ 3.722µs (SLO: <10.000µs 📉 -62.8%) vs baseline: +0.4% Memory: ✅ 42.605MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.0% ✅ replace_aspectTime: ✅ 212.367µs (SLO: <300.000µs 📉 -29.2%) vs baseline: +0.4% Memory: ✅ 43.116MB (SLO: <44.000MB -2.0%) vs baseline: +4.9% ✅ replace_noaspectTime: ✅ 5.258µs (SLO: <10.000µs 📉 -47.4%) vs baseline: -0.2% Memory: ✅ 42.526MB (SLO: <43.000MB 🟡 -1.1%) vs baseline: +3.6% ✅ repr_aspectTime: ✅ 1.382µs (SLO: <10.000µs 📉 -86.2%) vs baseline: ~same Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.3% ✅ repr_noaspectTime: ✅ 0.526µs (SLO: <10.000µs 📉 -94.7%) vs baseline: ~same Memory: ✅ 42.723MB (SLO: <43.000MB 🟡 -0.6%) vs baseline: +5.0% ✅ rstrip_aspectTime: ✅ 19.046µs (SLO: <20.000µs -4.8%) vs baseline: +0.5% Memory: ✅ 42.900MB (SLO: <43.000MB 🟡 -0.2%) vs baseline: +4.7% ✅ rstrip_noaspectTime: ✅ 1.983µs (SLO: <10.000µs 📉 -80.2%) vs baseline: +4.9% Memory: ✅ 42.664MB (SLO: <43.000MB 🟡 -0.8%) vs baseline: +4.3% ✅ slice_aspectTime: ✅ 15.935µs (SLO: <20.000µs 📉 -20.3%) vs baseline: +1.6% Memory: ✅ 42.979MB (SLO: <43.000MB 🟡 ~same) vs baseline: +5.0% ✅ slice_noaspectTime: ✅ 0.595µs (SLO: <10.000µs 📉 -94.1%) vs baseline: -0.2% Memory: ✅ 42.703MB (SLO: <43.000MB 🟡 -0.7%) vs baseline: +4.5% ✅ stringio_aspectTime: ✅ 54.688µs (SLO: <80.000µs 📉 -31.6%) vs baseline: +2.1% Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.9% ✅ stringio_noaspectTime: ✅ 3.595µs (SLO: <10.000µs 📉 -64.0%) vs baseline: +2.2% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.4% ✅ strip_aspectTime: ✅ 17.794µs (SLO: <20.000µs 📉 -11.0%) vs baseline: ~same Memory: ✅ 42.920MB (SLO: <43.000MB 🟡 -0.2%) vs baseline: +4.9% ✅ strip_noaspectTime: ✅ 1.824µs (SLO: <10.000µs 📉 -81.8%) vs baseline: -0.4% Memory: ✅ 42.802MB (SLO: <43.000MB 🟡 -0.5%) vs baseline: +4.7% ✅ swapcase_aspectTime: ✅ 18.517µs (SLO: <30.000µs 📉 -38.3%) vs baseline: +0.5% Memory: ✅ 42.998MB (SLO: <43.000MB 🟡 ~same) vs baseline: +4.7% ✅ swapcase_noaspectTime: ✅ 2.749µs (SLO: <10.000µs 📉 -72.5%) vs baseline: ~same Memory: ✅ 42.723MB (SLO: <43.000MB 🟡 -0.6%) vs baseline: +4.4% ✅ title_aspectTime: ✅ 18.238µs (SLO: <20.000µs -8.8%) vs baseline: +0.6% Memory: ✅ 42.979MB (SLO: <43.000MB 🟡 ~same) vs baseline: +4.9% ✅ title_noaspectTime: ✅ 2.657µs (SLO: <10.000µs 📉 -73.4%) vs baseline: +1.3% Memory: ✅ 42.684MB (SLO: <43.000MB 🟡 -0.7%) vs baseline: +4.5% ✅ translate_aspectTime: ✅ 23.952µs (SLO: <30.000µs 📉 -20.2%) vs baseline: 📈 +17.7% Memory: ✅ 42.939MB (SLO: <43.000MB 🟡 -0.1%) vs baseline: +4.9% ✅ translate_noaspectTime: ✅ 4.271µs (SLO: <10.000µs 📉 -57.3%) vs baseline: +0.2% Memory: ✅ 42.684MB (SLO: <43.000MB 🟡 -0.7%) vs baseline: +4.2% ✅ upper_aspectTime: ✅ 17.897µs (SLO: <30.000µs 📉 -40.3%) vs baseline: +0.6% Memory: ✅ 42.880MB (SLO: <43.000MB 🟡 -0.3%) vs baseline: +4.8% ✅ upper_noaspectTime: ✅ 2.397µs (SLO: <10.000µs 📉 -76.0%) vs baseline: +1.2% Memory: ✅ 42.782MB (SLO: <43.000MB 🟡 -0.5%) vs baseline: +4.4% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.258µs (SLO: <10.000µs 📉 -47.4%) vs baseline: 📈 +19.7% Memory: ✅ 41.465MB (SLO: <43.000MB -3.6%) vs baseline: +5.0% ✅ ospathbasename_noaspectTime: ✅ 4.390µs (SLO: <10.000µs 📉 -56.1%) vs baseline: -1.4% Memory: ✅ 41.406MB (SLO: <43.000MB -3.7%) vs baseline: +4.8% ✅ ospathjoin_aspectTime: ✅ 6.245µs (SLO: <10.000µs 📉 -37.6%) vs baseline: -2.6% Memory: ✅ 41.366MB (SLO: <43.000MB -3.8%) vs baseline: +4.9% ✅ ospathjoin_noaspectTime: ✅ 6.273µs (SLO: <10.000µs 📉 -37.3%) vs baseline: -2.6% Memory: ✅ 41.425MB (SLO: <43.000MB -3.7%) vs baseline: +5.0% ✅ ospathnormcase_aspectTime: ✅ 3.583µs (SLO: <10.000µs 📉 -64.2%) vs baseline: +1.7% Memory: ✅ 41.386MB (SLO: <43.000MB -3.8%) vs baseline: +4.9% ✅ ospathnormcase_noaspectTime: ✅ 3.634µs (SLO: <10.000µs 📉 -63.7%) vs baseline: +0.6% Memory: ✅ 41.406MB (SLO: <43.000MB -3.7%) vs baseline: +5.1% ✅ ospathsplit_aspectTime: ✅ 4.971µs (SLO: <10.000µs 📉 -50.3%) vs baseline: +0.3% Memory: ✅ 41.386MB (SLO: <43.000MB -3.8%) vs baseline: +4.7% ✅ ospathsplit_noaspectTime: ✅ 5.060µs (SLO: <10.000µs 📉 -49.4%) vs baseline: -1.3% Memory: ✅ 41.484MB (SLO: <43.000MB -3.5%) vs baseline: +5.0% ✅ ospathsplitdrive_aspectTime: ✅ 3.863µs (SLO: <10.000µs 📉 -61.4%) vs baseline: +1.4% Memory: ✅ 41.504MB (SLO: <43.000MB -3.5%) vs baseline: +5.1% ✅ ospathsplitdrive_noaspectTime: ✅ 0.742µs (SLO: <10.000µs 📉 -92.6%) vs baseline: -0.4% Memory: ✅ 41.366MB (SLO: <43.000MB -3.8%) vs baseline: +4.8% ✅ ospathsplitext_aspectTime: ✅ 4.740µs (SLO: <10.000µs 📉 -52.6%) vs baseline: +0.5% Memory: ✅ 41.465MB (SLO: <43.000MB -3.6%) vs baseline: +4.9% ✅ ospathsplitext_noaspectTime: ✅ 4.745µs (SLO: <10.000µs 📉 -52.6%) vs baseline: -1.0% Memory: ✅ 41.406MB (SLO: <43.000MB -3.7%) vs baseline: +5.0% 📈 iastaspectssplit - 12/12✅ rsplit_aspectTime: ✅ 1.714µs (SLO: <10.000µs 📉 -82.9%) vs baseline: 📈 +10.2% Memory: ✅ 41.425MB (SLO: <43.000MB -3.7%) vs baseline: +5.2% ✅ rsplit_noaspectTime: ✅ 1.664µs (SLO: <10.000µs 📉 -83.4%) vs baseline: +5.1% Memory: ✅ 41.465MB (SLO: <43.000MB -3.6%) vs baseline: +4.9% ✅ split_aspectTime: ✅ 1.598µs (SLO: <10.000µs 📉 -84.0%) vs baseline: +5.1% Memory: ✅ 41.504MB (SLO: <43.000MB -3.5%) vs baseline: +5.1% ✅ split_noaspectTime: ✅ 1.663µs (SLO: <10.000µs 📉 -83.4%) vs baseline: +6.8% Memory: ✅ 41.406MB (SLO: <43.000MB -3.7%) vs baseline: +5.0% ✅ splitlines_aspectTime: ✅ 1.548µs (SLO: <10.000µs 📉 -84.5%) vs baseline: +0.8% Memory: ✅ 41.445MB (SLO: <43.000MB -3.6%) vs baseline: +5.0% ✅ splitlines_noaspectTime: ✅ 1.601µs (SLO: <10.000µs 📉 -84.0%) vs baseline: +3.2% Memory: ✅ 41.386MB (SLO: <43.000MB -3.8%) vs baseline: +4.6% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.387µs (SLO: <20.000µs 📉 -83.1%) vs baseline: 📈 +11.9% Memory: ✅ 34.741MB (SLO: <35.500MB -2.1%) vs baseline: +5.0% ✅ 1-count-metrics-100-timesTime: ✅ 201.746µs (SLO: <220.000µs -8.3%) vs baseline: -2.7% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.7% ✅ 1-distribution-metric-1-timesTime: ✅ 3.369µs (SLO: <20.000µs 📉 -83.2%) vs baseline: -1.5% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.7% ✅ 1-distribution-metrics-100-timesTime: ✅ 219.132µs (SLO: <230.000µs -4.7%) vs baseline: -0.4% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.7% ✅ 1-gauge-metric-1-timesTime: ✅ 2.199µs (SLO: <20.000µs 📉 -89.0%) vs baseline: -0.6% Memory: ✅ 34.937MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.2% ✅ 1-gauge-metrics-100-timesTime: ✅ 136.471µs (SLO: <150.000µs -9.0%) vs baseline: -0.2% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +5.0% ✅ 1-rate-metric-1-timesTime: ✅ 3.178µs (SLO: <20.000µs 📉 -84.1%) vs baseline: -1.1% Memory: ✅ 34.760MB (SLO: <35.500MB -2.1%) vs baseline: +4.6% ✅ 1-rate-metrics-100-timesTime: ✅ 216.144µs (SLO: <250.000µs 📉 -13.5%) vs baseline: -2.5% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.8% ✅ 100-count-metrics-100-timesTime: ✅ 20.203ms (SLO: <22.000ms -8.2%) vs baseline: -3.2% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +5.3% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.278ms (SLO: <2.550ms 📉 -10.7%) vs baseline: -1.3% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.6% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.405ms (SLO: <1.550ms -9.4%) vs baseline: ~same Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ 100-rate-metrics-100-timesTime: ✅ 2.220ms (SLO: <2.550ms 📉 -12.9%) vs baseline: -2.2% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +5.2% ✅ flush-1-metricTime: ✅ 4.748µs (SLO: <20.000µs 📉 -76.3%) vs baseline: +1.8% Memory: ✅ 35.212MB (SLO: <35.500MB 🟡 -0.8%) vs baseline: +4.9% ✅ flush-100-metricsTime: ✅ 175.230µs (SLO: <250.000µs 📉 -29.9%) vs baseline: +0.3% Memory: ✅ 35.173MB (SLO: <35.500MB 🟡 -0.9%) vs baseline: +4.4% ✅ flush-1000-metricsTime: ✅ 2.179ms (SLO: <2.500ms 📉 -12.8%) vs baseline: ~same Memory: ✅ 36.038MB (SLO: <36.500MB 🟡 -1.3%) vs baseline: +4.8% 🟡 Near SLO Breach (15 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
No description provided.