Operate · 03

How fast is it?

The latest measured run, what it measures, and how to run it yourself.

The benchmark runs examples/goblin-pizza.ts, which uses methods, derived stats, timers, worker leases, retries and fencing. It starts its own fresh local cluster and never touches an existing one.

LATEST MEASURED RUN ·

A busy day in the garden.

Global customer calls / s
56,155
Read p99
12.9 ms
Write p99
228.3 ms
Independent Raft groups
8 × 3 replicas

Replica-local reads: lag is allowed. 70% reads / 30% mutations · HTTP/2 · 60.1 measured seconds.

Run passed. 8/8 group audits passed. 8 injected leader failures; quorum recovery 613–733 ms.

Apple M5 Pro; all replicas and load generators share one machine. Completed customer calls use the union measurement window; retries, worker traffic, and explicit replays do not inflate throughput. Reads may be stale; mutations and audits retain fresh checks.

Charts & every group →Raw measurements ↓

Run it yourself

Run and stress
npm ci
npm run build
cargo build --release --bin flower
npm run bench
cargo build --release --bin flower --bin flower-bench-driver
npm run bench:stress

# Customize the base command, with one copy of each flag:
npm run bench -- --duration 30 --concurrency 16 --workers 4 \
  --max-orders 64 --drain 180 --json bench/results/custom.json
npm run bench -- --help
  • What counts: successful customer calls (shop reads, orders, tips). A retried call counts once. Worker polls, deliveries, replays, setup and the audit don't count.
  • The workload is closed-loop: each customer waits for one call before starting the next. The mix is 30% orders, 40% reads and 30% tips until the order cap, then orders turn into reads.
  • Everything runs on one machine, servers and load generator together. Your production numbers depend on your workload and hardware.
  • The run fails on unexpected errors, an unfinished drain, bad fencing or replays, or a failed business check.
  • Reports go to bench/results/ as JSON and HTML, with the settings and binary used.

The base command starts one three-node group. bench:stress runs eight independent groups, each with its own customers, workers and leader crash. Its preset sets 256 customers per group, serial writer preparation and a 1,024-request writer queue, overriding the normal defaults. Environment variables override the preset. See the benchmark guide for all options, and the measurement analysis and CPU investigation for past results.

To publish a new result to this page, run inside nix develop, build first and leave profiling off:

Measure independent Raft groups
npm ci
cargo build --release --locked --bin flower --bin flower-bench-driver
npm run bench:stress
node scripts/publish-bench-results.mjs bench/results/latest.json
node scripts/publish-bench-results.mjs --check

Watch the live dashboard

The dashboard shows one tenant's stores, orders, oven timers, drone leases and rankings from a single watched pizza.dashboard({tenant}) value.

Run the live kitchen board
cargo build --release --bin flower
npm run demo:pizza
# Optional: fixed port, paused arrivals, stop automatically after two minutes
npm run demo:pizza -- --port 3030 --paused --duration 120

Open the printed URL. It starts a fresh three-node cluster with three tenants. Place orders, tip goblins, pause arrivals or crash the leader. Ctrl+C stops everything and deletes its data.

The dashboard reads replica-local data. It can lag with no bound, and a reconnect may show an older revision. The benchmark's shop reads use pizza.shop.local the same way; pass --read-consistency fresh to measure fresh reads instead.

Compare HTTP/1.1 and HTTP/2

Method calls use HTTP/1.1 by default. Add --http2 to use pooled HTTP/2. Node-to-node traffic always uses HTTP/2.

HTTP/2 methods with leader failover
npm run bench -- --http2 --duration 30 --concurrency 32 --workers 4 \
  --max-orders 96 --chaos --json bench/results/http2.json

Keep the workload and binary the same when you compare. The benchmark reuses request IDs when it retries; the HTTP/2 client itself doesn't retry uncertain calls.

Profile before tuning

Profile under the same mixed load before changing settings. Skip the leader crash for a first comparison.

macOS · native stack profile
CARGO_PROFILE_RELEASE_DEBUG=1 cargo build --release --bin flower
npm run bench -- --duration 30 --concurrency 32 --workers 4 \
  --max-orders 96 --drain 180 \
  --cpu-profile bench/results/profile-mixed.sample.txt \
  --json bench/results/profile-mixed.json

# Batch timing for the same mixed workload (one group).
FLOWER_PROFILE_STORAGE=1 FLOWER_PROFILE_EVALUATOR=1 \
node bench/profile-mixed.mjs --http2 --duration 30 --concurrency 128 \
  --workers 4 --max-orders 96 --json /tmp/flower-mixed-profile.json
  • The first command samples native stacks on the initial leader with macOS /usr/bin/sample. It shows where threads spend time, not CPU percentages or TypeScript lines.
  • The second writes an extra *-groups.json with per-batch timings. Phases overlap, so don't add them up as wall time.
  • node bench/profile-otel.mjs takes the same flags and writes an HTML report of request, writer, storage and Raft timing. See the OpenTelemetry guide.
  • Profiling slows things down. Rerun without it before quoting throughput.

More options are in the profiling commands.

Run the checks

Run the test suite from the checkout:

Validation from the checkout
npm run check
cargo test
cargo build
node tests/e2e.mjs
node tests/e2e-http2.mjs
node tests/e2e-watch.mjs

Benchmarks clean up their temporary data when they stop; pass --keep-data to keep it for inspection.