Skip to content

Commit 541fb8c

Browse files
authored
Update benchmarking.md
1 parent d843133 commit 541fb8c

File tree

1 file changed

+27
-26
lines changed

1 file changed

+27
-26
lines changed

content/learning-paths/servers-and-cloud-computing/flink-on-gcp/benchmarking.md

Lines changed: 27 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -8,44 +8,52 @@ layout: learningpathall
88

99

1010
## Apache Flink Benchmarking
11-
This guide provides step-by-step instructions to set up and run **Apache Flink Benchmarks** on a **GCP SUSE VMs**. It covers cloning the repository, building the benchmarks, exploring the JAR, and listing available benchmarks.
11+
This section walks you through running Apache Flink microbenchmarks on a Google Cloud Axion C4A (Arm64) SUSE VM. You will clone the official Flink-benchmarks repository, build the benchmark suite, explore available tests, and run the Remote Channel Throughput Benchmark, one of the key indicators of Flink’s communication and data-transfer performance.
1212

1313
### Clone the Repository
14-
Start by cloning the official Flink benchmarks repository. This repository contains all the benchmark definitions and example jobs.
14+
Clone the official Flink microbenchmarks repository:
1515

1616
```console
1717
cd ~
1818
git clone https://github.com/apache/flink-benchmarks.git
1919
cd flink-benchmarks
2020
```
21+
This repository contains microbenchmarks built using JMH (Java Microbenchmark Harness), widely used for JVM-level performance testing.
2122

2223
### Build the Benchmarks with Maven
23-
Use Maven to compile the benchmarks and generate the benchmark JAR. Skip tests to save time.
24+
Compile the benchmarks and create the executable JAR:
2425

2526
```console
2627
mvn clean package -DskipTests
2728
```
28-
- **mvn clean package** → Cleans previous builds and packages the project.
29+
What this does:
30+
* clean removes previous build artifacts
31+
* package compiles the code and produces benchmark JARs
32+
* -DskipTests speeds up the build since unit tests aren’t needed for benchmarking
33+
34+
After building, the compiled **benchmarks.jar** files appear under:
2935

30-
After this step, the target directory will contain the compiled **benchmarks.jar**.
36+
```output
37+
flink-benchmarks/target/
38+
```
3139

3240
### Explore the JAR Contents
33-
Verify the generated files inside the `target` directory:
41+
Verify that benchmarks.jar was generated:
3442

3543
```console
3644
cd target
3745
ls
3846
```
39-
You should see an output similar to:
47+
You should see:
4048

4149
```output
4250
benchmark-0.1.jar classes generated-test-sources maven-status protoc-plugins
4351
benchmarks.jar generated-sources maven-archiver protoc-dependencies test-classes
4452
```
45-
- **benchmarks.jar**→ The main benchmark JAR file used to run Flink benchmarks.
53+
benchmarks.jar — Contains all Flink microbenchmarks packaged with JMH.
4654

4755
### List Available Benchmarks
48-
To view all the benchmarks included in the JAR:
56+
View all benchmarks included in the JAR:
4957

5058
```console
5159
java -jar benchmarks.jar -l
@@ -54,13 +62,15 @@ java -jar benchmarks.jar -l
5462
- This helps you identify which benchmarks you want to execute on your VM.
5563

5664
### Run Selected Benchmarks
57-
While the Flink benchmarking project includes multiple suites for state backends, windowing, checkpointing, and scheduler performance, this Learning path focuses on the Remote Channel Throughput benchmark to evaluate network and I/O performance.
65+
While the Flink benchmarking project includes multiple suites for state backends, windowing, checkpointing, and scheduler performance, in this Learning path you will run the Remote Channel Throughput benchmark to evaluate network and I/O performance.
66+
67+
**Remote Channel Throughput**: This benchmark measures the data transfer rate between remote channels in Flink, helping to evaluate network and I/O performance.
5868

59-
**Remote Channel Throughput**: Measures the data transfer rate between remote channels in Flink, helping to evaluate network and I/O performance.
69+
Run the benchmark:
6070
```console
6171
java -jar benchmarks.jar org.apache.flink.benchmark.RemoteChannelThroughputBenchmark.remoteRebalance
6272
```
63-
You should see an output similar to:
73+
You should see output similar to:
6474
```output
6575
6676
Result "org.apache.flink.benchmark.RemoteChannelThroughputBenchmark.remoteRebalance":
@@ -83,25 +93,16 @@ RemoteChannelThroughputBenchmark.remoteRebalance DEBLOAT thrpt 30 10536.511
8393
- **Min Throughput**: The lowest throughput was observed, and it shows worst-case performance.
8494
- **Max Throughput**: Highest throughput observed, shows best-case performance.
8595

86-
### Benchmark summary on x86_64
87-
To compare the benchmark results, the following results were collected by running the same benchmark on a `x86 - c4-standard-4` (4 vCPUs, 15 GB Memory) x86_64 VM in GCP, running SUSE:
88-
89-
| Benchmark | Mode | Count | Score (ops/ms) | Error (±) | Min | Max | Stdev | CI (99.9%) | Units |
90-
|---------------------------------------------------|---------|-------|----------------|-----------|------------|------------|---------|------------------------|--------|
91-
| RemoteChannelThroughputBenchmark.remoteRebalance | ALIGNED | 30 | 24873.046 | 892.673 | 11195.028 | 12425.761 | 421.057 | [11448.649, 12011.275] | ops/ms |
92-
| RemoteChannelThroughputBenchmark.remoteRebalance | DEBLOAT | 30 | 11729.962 | 281.313 | 11195.028 | 12425.761 | 421.057 | [11448.649, 12011.275] | ops/ms |
93-
9496
### Benchmark summary on Arm64
95-
Results from the earlier run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in GCP (SUSE):
97+
Results from the run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in GCP (SUSE) are summarized below:
9698

9799
| Benchmark | Mode | Count | Score (ops/ms) | Error (±) | Min | Max | Stdev | CI (99.9%) | Units |
98100
|---------------------------------------------------|---------|-------|----------------|-----------|-----------|-----------|---------|------------------------|--------|
99101
| RemoteChannelThroughputBenchmark.remoteRebalance | ALIGNED | 30 | 17445.341 | 153.256 | 10289.593 | 10687.736 | 89.987 | [10476.390, 10596.633] | ops/ms |
100102
| RemoteChannelThroughputBenchmark.remoteRebalance | DEBLOAT | 30 | 10536.511 | 60.121 | 10289.593 | 10687.736 | 89.987 | [10476.390, 10596.633] | ops/ms |
101103

102-
### Apache Flink performance benchmarking comparison on Arm64 and x86_64
104+
### Apache Flink performance benchmarking observations on Arm64
103105

104-
- The **ALIGNED mode** achieved an average throughput of **17,445 ops/ms**, demonstrating higher performance on the Arm64 VM.
105-
- The **DEBLOAT mode** achieved an average throughput of **10,537 ops/ms**, slightly lower due to optimization differences.
106-
- The benchmark confirms that the **Arm64 architecture** efficiently handles Flink's remote channel throughput workloads.
107-
- Overall, the average throughput across both modes is approximately **14,854 ops/ms**, indicating strong baseline performance for Arm64 deployments.
106+
- Both the ALIGNED mode and DEBLOAT modes demonstrate a strong throughput on the Arm64 VM.
107+
- The benchmark confirms that the Arm64 architecture efficiently handles Flink's remote channel throughput workloads.
108+
- Overall, the average throughput across both modes is approximately **14,854 ops/ms**, indicating strong baseline performance for Arm64deployments.

0 commit comments

Comments
 (0)