This module contains microbenchmarks for Apache Pulsar.

Run benchmarks on Linux x86_64 when the numbers matter. That is Pulsar's most common deployment target, and results from elsewhere do not carry over. System.nanoTime() is far more expensive on macOS than on Linux, which skews the results in some cases — JMH's own measurement loop pays that cost on every invocation. async-profiler also supports only some of its sampling engines on macOS, so -prof async is less reliable there. Benchmarking on macOS or arm64 is fine while iterating — just confirm the result on Linux x86_64 before drawing a conclusion from it.

The benchmarks are written using JMH. To compile & run the benchmarks, use the following command:

# Compile everything including the shaded microbenchmarks jar
./gradlew :microbench:shadowJar

# run the benchmarks using the standalone shaded jar in any environment
java -jar microbench/build/libs/microbench-*-benchmarks.jar

Display help:

java -jar microbench/build/libs/microbench-*-benchmarks.jar -h

Listing all benchmarks:

java -jar microbench/build/libs/microbench-*-benchmarks.jar -l

Running specific benchmarks:

java -jar microbench/build/libs/microbench-*-benchmarks.jar ".*BenchmarkName.*"

Running specific benchmarks with machine-readable output and saving the output to a file:

ts=$(date +%s)
java -jar microbench/build/libs/microbench-*-benchmarks.jar -rf json -rff jmh-result-$ts.json ".*BenchmarkName.*" | tee jmh-result-$ts.txt

The jmh-result-*.json file can be used to visualize the results using JMH Visualizer.

Checking what benchmarks match the pattern:

java -jar microbench/build/libs/microbench-*-benchmarks.jar ".*BenchmarkName.*" -lp

Profiling benchmarks with async-profiler:

Set LIBASYNCPROFILER_PATH to the path of the async-profiler library.

Corretto JDK ships with async-profiler (asprof binary and libasyncProfiler dynamic library)

LIBASYNCPROFILER_PATH=$(ls $JAVA_HOME/lib/libasyncProfiler.*)

Alternatively, download async-profiler from https://github.com/async-profiler/async-profiler/releases and install to ~/async-profiler directory.

Mac OS example:

LIBASYNCPROFILER_PATH=$HOME/async-profiler/lib/libasyncProfiler.dylib

Linux example:

LIBASYNCPROFILER_PATH=$HOME/async-profiler/lib/libasyncProfiler.so

Then run the benchmarks with the -prof argument:

java -jar microbench/build/libs/microbench-*-benchmarks.jar -prof async:libPath=$LIBASYNCPROFILER_PATH\;output=flamegraph\;dir=profile-results ".*BenchmarkName.*"

The default value for event is cpu, which is a request for the best available CPU sampling engine rather than a specific one, so what it resolves to depends on the platform. If the profiler fails to start, add \;event=itimer to the -prof argument: itimer is available everywhere.

It's possible to add options to the async-profiler that aren't supported by the JMH async-profiler plugin. This can be done by adding rawCommand option to the -prof argument. This example shows how to add all (new in Async Profiler 4.1), jfrsync (record JFR events such as garbage collection) and cstack=vmx options.

java -jar microbench/build/libs/microbench-*-benchmarks.jar -prof async:libPath=$LIBASYNCPROFILER_PATH\;output=jfr\;dir=profile-results\;rawCommand=all,jfrsync,cstack=vmx ".*BenchmarkName.*"

Outside Linux this particular command needs \;event=itimer as well. all turns on wall clock profiling, and where the cpu engine falls back to the wall clock engine the profiler refuses to start with Cannot start wall clock with the selected event, which shows up as a <failure> on the first warmup iteration and an empty result directory.

output=jfr writes one recording per benchmark, into a directory named after the benchmark under dir=. The jfrFlamegraphs Gradle task renders each recording into every view at once — point it at the whole output directory and it finds the recordings inside:

./gradlew jfrFlamegraphs -Pjfr=profile-results

Each recording gets a directory beside it named after the file without its extension plus a -flamegraphs suffix, holding cpu, wall, alloc and lock, each rendered merged (cpu.html), split per thread (cpu_threads.html) and grouped into async-profiler's categories (cpu_classify.html). A view whose event the recording does not contain is skipped. See Performance testing for the analysis workflow.

The .jfr can also be opened in Eclipse Mission Control or IntelliJ IDEA, or handed to an AI agent through the Jafar MCP server, which lets the agent query the recording directly with tools such as jfr_diagnose and jfr_stackprofile — see Jafar MCP analysis.