Add the protocol-comparison module
This commit is contained in:
74
protocol-comparison/docs/02-benchmarks.md
Normal file
74
protocol-comparison/docs/02-benchmarks.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# 2. How they measure
|
||||
|
||||
Previous: [1. Three protocols](01-three-protocols.md) · Next: [3. Payload](03-payload.md)
|
||||
|
||||
---
|
||||
|
||||
## Read this before the numbers
|
||||
|
||||
Client and server are **the same process on one machine**, talking over loopback: two cores,
|
||||
3.9 GB, JDK 25, Boot 4.1.1. That has three consequences, and ignoring them makes the numbers
|
||||
say the opposite of the truth.
|
||||
|
||||
1. **There is no network.** On a real link, a 35-byte message and an 83-byte message differ by
|
||||
more than the framing cost, and protobuf's size advantage starts paying. Here it does not.
|
||||
2. **A protocol that does less looks faster.** WebSocket wins both benchmarks below partly because
|
||||
it has no flow control, no deadlines, no schema and no per-call metadata. That is a real cost
|
||||
difference, and it is also exactly what you give up.
|
||||
3. **Ratios, not absolutes.** Absolute microsecond figures from a two-core sandbox mean nothing
|
||||
for your hardware. The ordering and the rough factors are what transfer.
|
||||
|
||||
Re-measured on every `./scripts/run-all.sh`, and re-measured **in their own JVM invocation**.
|
||||
That is not tidiness. The back-pressure tests in [chapter 4](04-backpressure.md) leave unbounded
|
||||
producers spinning, and running the throughput benchmark after them on a two-core box halved
|
||||
every number — gRPC fell from 44 801 msgs/s to 18 477. If you take one operational lesson from
|
||||
this module rather than a protocol one, let it be that: a benchmark that shares a machine with
|
||||
anything is measuring the machine.
|
||||
|
||||
## Request/response
|
||||
|
||||
5 000 sequential calls after 2 000 warm-up
|
||||
([`request-response.txt`](output/request-response.txt)):
|
||||
|
||||
```
|
||||
gRPC n=5000 p50= 291.3 us p99= 1015.3 us mean= 324.4 us ~3,083 calls/s
|
||||
RSocket n=5000 p50= 300.9 us p99= 1337.4 us mean= 350.9 us ~2,850 calls/s
|
||||
WebSocket n=5000 p50= 124.3 us p99= 1447.8 us mean= 172.8 us ~5,786 calls/s
|
||||
```
|
||||
|
||||
**gRPC and RSocket are the same speed.** Within noise across runs the two swap places; treat them
|
||||
as indistinguishable for request/response on a warm connection.
|
||||
|
||||
**Raw WebSocket is about twice as fast at the median — and has the worst tail.** p50 of
|
||||
124 µs against a p99 of 1 448 µs is a 12x spread. gRPC's is 3.5x. The
|
||||
median is cheap because a text frame over an already-open socket is nearly free; the tail is bad
|
||||
because nothing is scheduling anything, so a GC pause or a scheduling hiccup lands on the request
|
||||
undiluted.
|
||||
|
||||
If your SLO is a percentile rather than an average — and it should be — that table
|
||||
does not say what a first glance suggests.
|
||||
|
||||
## Server streaming
|
||||
|
||||
50 000 messages on one connection
|
||||
([`stream-throughput.txt`](output/stream-throughput.txt)):
|
||||
|
||||
```
|
||||
gRPC 50,000 msgs in 1.116 s = 44,801 msgs/s ( 22.32 us each)
|
||||
RSocket 50,000 msgs in 0.702 s = 71,175 msgs/s ( 14.05 us each)
|
||||
WebSocket 50,000 msgs in 0.459 s = 109,022 msgs/s ( 9.17 us each)
|
||||
```
|
||||
|
||||
Same ordering, wider gaps: WebSocket about 2.4x gRPC, RSocket about 1.6x.
|
||||
|
||||
Now read the WebSocket row again with its harness in view. The client's inbox is an
|
||||
`ArrayBlockingQueue` sized to hold **all fifty thousand messages**, because there was nothing else
|
||||
to do with them — a raw WebSocket consumer that falls behind can buffer or drop, and those
|
||||
are the only two options. WebSocket "won" this benchmark by buffering the entire stream in the
|
||||
client's heap.
|
||||
|
||||
That is not a rhetorical point. It is the next chapter.
|
||||
|
||||
---
|
||||
|
||||
Previous: [1. Three protocols](01-three-protocols.md) · Next: [3. Payload](03-payload.md)
|
||||
Reference in New Issue
Block a user