3.5 KiB
2. How they measure
Previous: 1. Three protocols · Next: 3. Payload
Read this before the numbers
Client and server are the same process on one machine, talking over loopback: two cores, 3.9 GB, JDK 25, Boot 4.1.1. That has three consequences, and ignoring them makes the numbers say the opposite of the truth.
- There is no network. On a real link, a 35-byte message and an 83-byte message differ by more than the framing cost, and protobuf's size advantage starts paying. Here it does not.
- A protocol that does less looks faster. WebSocket wins both benchmarks below partly because it has no flow control, no deadlines, no schema and no per-call metadata. That is a real cost difference, and it is also exactly what you give up.
- Ratios, not absolutes. Absolute microsecond figures from a two-core sandbox mean nothing for your hardware. The ordering and the rough factors are what transfer.
Re-measured on every ./scripts/run-all.sh, and re-measured in their own JVM invocation.
That is not tidiness. The back-pressure tests in chapter 4 leave unbounded
producers spinning, and running the throughput benchmark after them on a two-core box halved
every number — gRPC fell from 44 801 msgs/s to 18 477. If you take one operational lesson from
this module rather than a protocol one, let it be that: a benchmark that shares a machine with
anything is measuring the machine.
Request/response
5 000 sequential calls after 2 000 warm-up
(request-response.txt):
gRPC n=5000 p50= 291.3 us p99= 1015.3 us mean= 324.4 us ~3,083 calls/s
RSocket n=5000 p50= 300.9 us p99= 1337.4 us mean= 350.9 us ~2,850 calls/s
WebSocket n=5000 p50= 124.3 us p99= 1447.8 us mean= 172.8 us ~5,786 calls/s
gRPC and RSocket are the same speed. Within noise across runs the two swap places; treat them as indistinguishable for request/response on a warm connection.
Raw WebSocket is about twice as fast at the median — and has the worst tail. p50 of 124 µs against a p99 of 1 448 µs is a 12x spread. gRPC's is 3.5x. The median is cheap because a text frame over an already-open socket is nearly free; the tail is bad because nothing is scheduling anything, so a GC pause or a scheduling hiccup lands on the request undiluted.
If your SLO is a percentile rather than an average — and it should be — that table does not say what a first glance suggests.
Server streaming
50 000 messages on one connection
(stream-throughput.txt):
gRPC 50,000 msgs in 1.116 s = 44,801 msgs/s ( 22.32 us each)
RSocket 50,000 msgs in 0.702 s = 71,175 msgs/s ( 14.05 us each)
WebSocket 50,000 msgs in 0.459 s = 109,022 msgs/s ( 9.17 us each)
Same ordering, wider gaps: WebSocket about 2.4x gRPC, RSocket about 1.6x.
Now read the WebSocket row again with its harness in view. The client's inbox is an
ArrayBlockingQueue sized to hold all fifty thousand messages, because there was nothing else
to do with them — a raw WebSocket consumer that falls behind can buffer or drop, and those
are the only two options. WebSocket "won" this benchmark by buffering the entire stream in the
client's heap.
That is not a rhetorical point. It is the next chapter.
Previous: 1. Three protocols · Next: 3. Payload