Skip to content

Conversation

@aviralgarg05
Copy link

Description

This commit addresses issue #14111 by documenting the investigation into performance change detection tooling for the OpenTelemetry Collector.

Changes:

  • Created comprehensive performance-benchmarking.md guide covering:
    • Running benchmarks locally (make gobenchmark, timebenchmark)
    • Writing benchmark tests following Go conventions
    • CodSpeed continuous performance monitoring integration
    • Best practices for performance-sensitive code changes
    • Troubleshooting common benchmark issues
  • Updated CONTRIBUTING.md with Performance Testing section
  • Added changelog entry documenting the enhancement

The investigation found that CodSpeed (integrated in PR #14160) provides continuous performance regression detection on all PRs and commits to main, with <1% variance and commit-level differential flamegraphs.

This documentation enables contributors to understand the performance testing infrastructure and follow best practices when making changes that could affect collector performance.

Link to tracking issue

Fixes #14111

Testing

NA

Documentation

Created Performance Benchmarking Guide
File: docs/performance-benchmarking.md
Updated CONTRIBUTING.md
File: CONTRIBUTING.md

@aviralgarg05 aviralgarg05 requested a review from a team as a code owner November 29, 2025 10:47
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Nov 29, 2025

CLA Signed

The committers listed above are authorized under a signed CLA.

@dmathieu dmathieu added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Dec 1, 2025
Address maintainer review feedback for issue open-telemetry#14111.

Changes to docs/performance-benchmarking.md:
- Fix order: MEMBENCH environment variable is set before running benchmarks
- Fix timebenchmark description to explain it skips fast-running benchmarks
- Update benchmark examples to use b.Loop() instead of for i := 0; i < b.N; i++
- Remove b.ResetTimer() as it is not necessary with b.Loop()
- Fix trailing whitespace

Other changes:
- Remove changelog entry (not needed for documentation changes)
@aviralgarg05 aviralgarg05 force-pushed the docs/add-performance-benchmarking-guide branch from 6b93093 to cf33fe2 Compare December 1, 2025 09:02
@dmathieu
Copy link
Member

dmathieu commented Dec 1, 2025

Please avoid force-pushing PRs. That breaks the GitHub review UI.

@aviralgarg05
Copy link
Author

Please avoid force-pushing PRs. That breaks the GitHub review UI.

Sorry! will keep in mind next time

@codspeed-hq
Copy link

codspeed-hq bot commented Dec 1, 2025

CodSpeed Performance Report

Merging #14234 will improve performances by ×2.9

Comparing aviralgarg05:docs/add-performance-benchmarking-guide (8021aaa) with main (7012862)

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

⚡ 2 improvements
✅ 71 untouched

Benchmarks breakdown

Benchmark BASE HEAD Change
zstdWithConcurrency 19 µs 6.6 µs ×2.9
BenchmarkSplittingBasedOnItemCountManySmallLogs 3.3 ms 2.4 ms +35.11%

@codecov
Copy link

codecov bot commented Dec 1, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.16%. Comparing base (7012862) to head (8021aaa).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #14234      +/-   ##
==========================================
- Coverage   92.16%   92.16%   -0.01%     
==========================================
  Files         668      668              
  Lines       41463    41463              
==========================================
- Hits        38216    38214       -2     
- Misses       2214     2215       +1     
- Partials     1033     1034       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

… docs

Added the following terms to the spell-check allowed words list:
- benchtime: Go benchmark time flag
- cpuprofile: Go CPU profiling flag
- flamegraphs: Performance visualization tool term
- MEMBENCH: Memory benchmark environment variable
- memprofile: Go memory profiling flag
- timebenchmark: Custom benchmark timing function reference
- walltime: Wall-clock time measurement term

These terms are used in the new performance benchmarking documentation
and are legitimate technical vocabulary.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Skip Changelog PRs that do not require a CHANGELOG.md entry

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate tooling to help identify performance changes

2 participants