commit 8547981634cb62bfeed9c4578472143e9811d246 Author: Ankur Date: Fri Jul 31 22:53:13 2026 +0530 Spring gRPC on Spring Boot 4: four call types, exception mapping, and a symptom-first troubleshooting suite Companion code for the ankurm.com guide. Verified on Spring Boot 4.1.0, spring-grpc 1.1.0, grpc-java 1.80.0, protobuf-java 4.34.2, JDK 25.0.3. results-full.txt is unedited mvn test output: 11 tests, 0 failures. No installs needed - protoc and the gRPC codegen plugin resolve as Maven artifacts, and every test uses the in-process transport. _01_basics all four call types and how their failure modes differ: iterator semantics on server streaming, the half-close that client streaming hangs without, and the independence of the two streams in bidi. _02_troubleshooting four failures reproduced then fixed - the 4 MB message limit and which side enforces it, the absent default deadline, cancellation that never interrupts a thread, and errors that arrive as UNKNOWN with no description. _03_exceptions @GrpcAdvice / @GrpcExceptionHandler: domain exception to NOT_FOUND with trailers, validation to INVALID_ARGUMENT, and a catch-all that returns a deliberate INTERNAL without leaking the original message across the boundary. docs/01 symptom-first troubleshooting index, each entry marked [tested] or [documented] docs/02 complete spring.grpc.* property reference, and the split between the Boot 4.1 integration (org.springframework.boot, version 4.1.0, owns properties and auto-configuration) and the Spring gRPC project (org.springframework.grpc, version 1.1.0, owns the programming model). The older standalone spring-grpc-spring-boot-starter stops at 1.0.3 and is what most search results describe. Findings worth the commit message - The in-process transport CANNOT enforce message size limits: it passes messages by reference and never serialises them. A 4 MB + 1 KB message goes through cleanly in tests and fails in production with RESOURCE_EXHAUSTED. The test asserts this rather than pretending otherwise. Same blind spot covers compression, TLS, keepalive and LB. - Two property names cost real time while writing this: spring.grpc.server.inprocess.name (not in-process) and spring.grpc.client.channel..target (singular channel, and target not address). The second failure surfaces as UnknownHostException on the channel NAME. - gRPC still has no default deadline, and cancellation only sets a Context flag. diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..35344dd --- /dev/null +++ b/.gitignore @@ -0,0 +1,11 @@ +target/ +*.class +t1.txt +t2.txt +gen.txt +.idea/ +*.iml +.vscode/ +.DS_Store + +# results-full.txt IS committed on purpose -- it is the evidence for the blog post diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..aa5473f --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Ankur Mhatre + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..031abf9 --- /dev/null +++ b/README.md @@ -0,0 +1,100 @@ +# Spring gRPC with Spring Boot 4 + +Companion repository for **[Spring gRPC with Spring Boot 4](https://ankurm.com/spring-grpc-spring-boot-4/)** +on [ankurm.com](https://ankurm.com). + +The post covers the concepts. This repository adds the part that is hard to find written down: a +symptom-first troubleshooting guide, with the failures reproduced as tests wherever a test can +honestly reproduce them. + +Verified against **Spring Boot 4.1.0**, **spring-grpc 1.1.0**, **grpc-java 1.80.0**, +**protobuf-java 4.34.2**, **JDK 25.0.3**. Full output in [`results-full.txt`](results-full.txt). + +--- + +## Quick start + +Nothing to install — `protoc` and the gRPC codegen plugin are resolved as Maven artifacts, and every +test runs over the in-process transport, so no port is bound. + +```console +git clone https://ankurm.com/git.app/asmhatre/spring-grpc-boot4.git +cd spring-grpc-boot4 +mvn test +``` + +8 tests, a few seconds. + +--- + +## What is here + +| Test | Covers | +|---|---| +| [`FourCallTypesTest`](src/test/java/com/ankurm/grpc/_01_basics/FourCallTypesTest.java) | Unary, server streaming, client streaming, bidirectional — and how their failure modes differ | +| [`HardToDiagnoseTest`](src/test/java/com/ankurm/grpc/_02_troubleshooting/HardToDiagnoseTest.java) | Four failures reproduced and fixed: message size limits, missing deadlines, unobserved cancellation, useless error statuses | +| [`ExceptionMappingTest`](src/test/java/com/ankurm/grpc/_03_exceptions/ExceptionMappingTest.java) | `@GrpcAdvice` / `@GrpcExceptionHandler` — domain exception → `NOT_FOUND`, validation → `INVALID_ARGUMENT`, catch-all → `INTERNAL` without leaking detail | + +| Document | Covers | +|---|---| +| [Troubleshooting](docs/01-troubleshooting.md) | Symptom-first index: every entry starts from the error message you actually see | +| [Properties and versions](docs/02-properties-and-versions.md) | Complete `spring.grpc.*` reference, and which of the two projects owns what | + +### Exception mapping + +[`OrderExceptionAdvice`](src/main/java/com/ankurm/grpc/orders/OrderExceptionAdvice.java) is the gRPC +analogue of `@RestControllerAdvice`. The service throws plain domain exceptions and never mentions +`Status`: + +```java +@GrpcAdvice +public class OrderExceptionAdvice { + + @GrpcExceptionHandler(OrderNotFoundException.class) + public StatusException handleNotFound(OrderNotFoundException ex) { + Metadata trailers = new Metadata(); + trailers.put(REASON_KEY, "ORDER_NOT_FOUND"); + return Status.NOT_FOUND.withDescription(ex.getMessage()).asException(trailers); + } +} +``` + +Without it, every domain failure arrives as `UNKNOWN` with a **null description** — no code to +branch on and no message to read. + +The service itself ([`OrderServiceImpl`](src/main/java/com/ankurm/grpc/orders/OrderServiceImpl.java)) +is heavily commented and is where the correct patterns live — cancellation checks, half-close +handling, `StreamObserver` thread-safety, structured errors. + +--- + +## Five findings + +1. **gRPC is a first-class Boot starter now, and "Spring gRPC" means two things.** + `org.springframework.boot:spring-boot-starter-grpc-server` (version **4.1.0**) owns + auto-configuration and every `spring.grpc.*` property; `org.springframework.grpc:spring-grpc-core` + (version **1.1.0**) owns the programming model — `@GrpcService`, `@GrpcAdvice`, + `GrpcChannelFactory`. The older standalone `spring-grpc-spring-boot-starter` stops at 1.0.3 and + is what most search results describe. See [docs/02](docs/02-properties-and-versions.md). +2. **The in-process transport cannot enforce message size limits.** It passes messages by reference + and never serialises them, so a 4 MB + 1 KB message sails through — and fails in production with + `RESOURCE_EXHAUSTED`. Asserted in the test suite. The same blind spot covers compression, TLS, + keepalive and load balancing. +3. **Two property names that cost real time.** `spring.grpc.server.inprocess.name` (not + `in-process`) and `spring.grpc.client.channel..target` (singular `channel`, and `target` not + `address`). Getting the second wrong yields `UnknownHostException` on the channel *name*, which + sends you to look at DNS. +4. **gRPC has no default deadline.** A call without one waits forever. This is the single most + common cause of a gRPC service that silently stops responding. +5. **Cancellation does not interrupt your thread.** It sets a `Context` flag. A server that never + checks it keeps working for a client that left ten minutes ago. + +--- + +## Reference machine + +AMD Ryzen 5 5600U, Windows 11, `java 25.0.3+9-LTS-195`, Maven 3.9.9. + +## Licence + +MIT. See [LICENSE](LICENSE). diff --git a/docs/01-troubleshooting.md b/docs/01-troubleshooting.md new file mode 100644 index 0000000..7bf9c78 --- /dev/null +++ b/docs/01-troubleshooting.md @@ -0,0 +1,188 @@ +# Troubleshooting Spring gRPC on Boot 4 + +Symptom-first. Each entry is something whose error message points somewhere other than its cause. +Entries marked **[tested]** are reproduced in +[`HardToDiagnoseTest`](../src/test/java/com/ankurm/grpc/_02_troubleshooting/HardToDiagnoseTest.java); +entries marked **[documented]** need a real network and are stated rather than asserted, because a +test claiming to prove them over in-process transport would be lying. + +--- + +## `UNAVAILABLE: Unable to resolve host ` — **[tested]** + +**Cause, 90% of the time: a typo in a property name**, not a networking problem. + +An unmatched channel name is passed straight to the DNS resolver as a target. Boot does not warn +that the channel is unconfigured. The exact names, both of which are easy to get wrong: + +```properties +spring.grpc.server.inprocess.name=orders-test # NOT "in-process" +spring.grpc.client.channel.orders.target=static://... # SINGULAR "channel"; "target", NOT "address" +``` + +Also check `spring.grpc.client.inprocess.enabled=true` if you are using the in-process transport +from the client side. + +## `IllegalStateException: No grpc channel factory found that supports target : ` + +`GrpcChannelFactory` is a composite that asks each registered factory `supports(target)` **before** +named-channel indirection is applied. Passing a bare logical name to `createChannel()` can therefore +fail even though the name is correctly configured. In application code, inject the stub (via +`@ImportGrpcClients`) and let Boot resolve the target; in tests, pass the full target. + +## `RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 4194304` — **[tested]** + +The default limit is **4 MB, per message, on the receiver**, and the two sides are configured +separately. + +```properties +# large RESPONSE -> the CLIENT is the receiver +spring.grpc.client.channel..inbound.message.max-size=16MB +# large REQUEST -> the SERVER is the receiver +spring.grpc.server.inbound.message.max-size=16MB +``` + +Per call: `stub.withMaxInboundMessageSize(16 * 1024 * 1024)`. + +Half of all "I raised the limit and it still fails" reports are the asymmetry: raising it on the +server does nothing for a large response. + +> **The testing trap.** The in-process transport passes messages **by reference and never +> serialises them**, so it *cannot* enforce size limits. A payload-size regression passes every +> in-process test you have and fails in production. The same applies to compression and anything +> else depending on the wire format. This is demonstrated — the test asserts that a 4 MB + 1 KB +> message goes through in-process without complaint. + +Prefer streaming to large unary messages: the limit is per message, so 1,000 small messages are +fine. Do not raise the limit globally — a large limit turns a malformed request into an OOM. + +## Calls that hang forever — **[tested]** + +**gRPC has no default deadline.** A call without one waits indefinitely. Nothing warns you, and in +testing the server always responds quickly, so it surfaces the day something downstream is slow — +as a thread pool that fills up with no errors in the log. + +```java +stub.withDeadlineAfter(2, TimeUnit.SECONDS).getOrder(request); +``` + +Per channel: `spring.grpc.client.channel..default.deadline=2s`, or register a +`DefaultDeadlineSetupClientInterceptor`. + +Deadlines are **absolute and propagate**: if A calls B with 2s remaining, B sees 2s, not a fresh 2s. +Set the deadline at the edge; do not re-set it at every hop, or a deep chain multiplies its budget. + +## The server keeps working after the client gave up — **[tested]** + +gRPC **does not interrupt your thread** on cancellation. It sets a flag on the `Context`. + +```java +for (Item item : items) { + if (Context.current().isCancelled()) return; // no onCompleted/onError -- stream is closed + responseObserver.onNext(convert(item)); +} +``` + +For blocking work handed to another thread, propagate with `Context.current().wrap(runnable)` — +otherwise the flag is invisible there. Calling `onNext`/`onCompleted` after cancellation throws +`IllegalStateException`. + +## Every error is `UNKNOWN` with no message — **[tested]** + +Any exception escaping a handler becomes `UNKNOWN` with a **null description** — deliberately, since +leaking exception text across a service boundary is an information-disclosure risk. + +```java +Metadata trailers = new Metadata(); +trailers.put(REASON_KEY, "ORDER_LOCKED"); +responseObserver.onError(new StatusRuntimeException( + Status.FAILED_PRECONDITION.withDescription("order is locked"), trailers)); +``` + +Choose the code carefully — it is the contract that retry policies, circuit breakers and dashboards +key off: + +| Do **not** retry | Retry may help | +|---|---| +| `INVALID_ARGUMENT`, `NOT_FOUND`, `ALREADY_EXISTS`, `FAILED_PRECONDITION`, `PERMISSION_DENIED`, `UNAUTHENTICATED` | `UNAVAILABLE`, `DEADLINE_EXCEEDED`, `RESOURCE_EXHAUSTED`, `ABORTED` | + +Getting this wrong makes non-retryable failures look retryable and turns one bad request into a +storm. + +## `IllegalStateException: call already closed` / `half-closed` + +You called `onNext` or `onCompleted` after the stream ended. Usual causes: + +- responding after cancellation or deadline expiry (check `Context.current().isCancelled()` first); +- calling `onCompleted()` twice; +- calling `onError()` and then `onCompleted()` — `onError` is terminal; +- two threads writing to one `StreamObserver`. **`StreamObserver` is not thread-safe.** Concurrent + `onNext` corrupts the stream, and the symptom is usually a deserialization error on the *far* + side, pointing nowhere near the bug. + +## A client-streaming call never completes + +You forgot the half-close. `requestObserver.onCompleted()` is what tells the server "no more +requests"; without it the server's `onCompleted` never fires and the call hangs until the deadline — +or forever, if you did not set one. + +## Retries do nothing — **[documented]** + +gRPC retries are configured through the **service config**, not a client API, and are off by +default: + +```properties +spring.grpc.client.channel.orders.service-config.methodConfig[0].retryPolicy.maxAttempts=4 +spring.grpc.client.channel.orders.service-config.methodConfig[0].retryPolicy.initialBackoff=0.5s +spring.grpc.client.channel.orders.service-config.methodConfig[0].retryPolicy.retryableStatusCodes[0]=UNAVAILABLE +``` + +Three reasons they silently do nothing: + +1. The status you are failing with is not in `retryableStatusCodes`. +2. The response was already partially delivered — gRPC will not retry a committed stream. +3. Retry is disabled on the channel (`enableRetry()` / the corresponding property). + +Only make idempotent methods retryable. gRPC has no idea whether your RPC is safe to repeat. + +## Connections drop every N minutes through a proxy — **[documented]** + +Load balancers and ingress controllers close idle HTTP/2 connections. gRPC reuses one long-lived +connection, so an idle stream is not an idle *connection* — the proxy disagrees. + +```properties +spring.grpc.client.channel..keepalive.time=30s +spring.grpc.client.channel..keepalive.timeout=5s +spring.grpc.server.keepalive.connection.max-idle-time=... +spring.grpc.server.keepalive.connection.max-age=... +``` + +Keep the client's keepalive interval **shorter** than the proxy's idle timeout. Note the server can +reject keepalives that are too frequent (`ENHANCE_YOUR_CALM` / `too_many_pings`), which looks like a +random disconnect — so both ends need to agree. + +## All traffic goes to one backend — **[documented]** + +The default load-balancing policy is `pick_first`, not `round_robin`. With a DNS target resolving to +several addresses, a gRPC client picks one connection and keeps it. This is correct behaviour and +almost never what people expect behind a headless Kubernetes service. + +Set `round_robin` via service config, and remember that DNS re-resolution happens on connection +failure, not on a timer — so scaling up does not redistribute existing connections. + +--- + +## Test transport limitations, collected + +The in-process transport is excellent for testing protocol behaviour and misleading for anything +wire-related. It **does** exercise interceptors, metadata, status codes, deadlines, cancellation and +flow control. It does **not** exercise: + +- message size limits (no serialization — proven in the test suite), +- compression, +- TLS and credentials negotiation, +- keepalive, GOAWAY, idle timeouts, +- name resolution and load balancing, +- HTTP/2 framing and header limits. + +Anything in that second list needs a real port, and belongs in a smaller, slower test tier. diff --git a/docs/02-properties-and-versions.md b/docs/02-properties-and-versions.md new file mode 100644 index 0000000..f718ad5 --- /dev/null +++ b/docs/02-properties-and-versions.md @@ -0,0 +1,144 @@ +# Properties reference, and which project owns what + +Two things that cause disproportionate confusion: the exact property names, and the fact that +**"Spring gRPC" now means two different things with two different version numbers**. + +--- + +## 1. Boot 4.1 integration vs. the Spring gRPC project + +These are separate artifacts with separate versions, and both are on your classpath. + +| | **Spring Boot gRPC integration** | **Spring gRPC** | +|---|---|---| +| Group | `org.springframework.boot` | `org.springframework.grpc` | +| Artifacts | `spring-boot-starter-grpc-server`, `spring-boot-starter-grpc-client`, `spring-boot-grpc-server`, `spring-boot-grpc-client`, `spring-boot-grpc-test` | `spring-grpc-core` | +| Version here | **4.1.0** (the Boot version) | **1.1.0** | +| Owns | auto-configuration, `spring.grpc.*` properties, starters, actuator/health, observation wiring | the programming model: `@GrpcService`, `@GrpcAdvice`, `@GrpcExceptionHandler`, `GrpcChannelFactory`, `@ImportGrpcClients`, interceptor infrastructure | +| Managed by | the Boot BOM directly | the Boot BOM, via the `spring-grpc.version` property | + +Practical consequences: + +- **Property names (`spring.grpc.*`) come from Boot 4.1**, defined in `GrpcServerProperties` / + `GrpcClientProperties` under `org.springframework.boot.grpc.*.autoconfigure`. Version them against + your Boot version. +- **Annotations and interfaces come from Spring gRPC 1.1.0**, in `org.springframework.grpc.*`. + Version them against `spring-grpc.version`. +- Boot 4.1 also pins `grpc-java` (**1.80.0** here) and `protobuf-java` (**4.34.2**). Both are + older than the newest releases on Maven Central — that is deliberate, and overriding them + independently is how you get `NoSuchMethodError` between grpc-java and its Netty shading. + +### The older path, and why you will see it in search results + +Before Boot 4, the community project shipped its own starter: + +```xml + + + org.springframework.grpc + spring-grpc-spring-boot-starter + +``` + +That artifact stops at **1.0.3** and is not what Boot 4 uses. On Boot 4 use the Boot starters and +let the BOM manage everything: + +```xml + + + org.springframework.boot + spring-boot-starter-grpc-server + + + org.springframework.boot + spring-boot-starter-grpc-client + +``` + +Most blog posts and Stack Overflow answers you find will be describing the older artifact. The +programming model is largely the same; the dependency coordinates and the property names are not. + +--- + +## 2. Server properties — exact names + +Read from `spring-boot-grpc-server-4.1.0.jar`'s configuration metadata. This is the complete list. + +| Property | Notes | +|---|---| +| `spring.grpc.server.enabled` | | +| `spring.grpc.server.port` | `-1` disables the network server (in-process only); `0` picks an ephemeral port | +| `spring.grpc.server.address` | | +| `spring.grpc.server.inprocess.name` | **`inprocess`, one word — not `in-process`** | +| `spring.grpc.server.inbound.message.max-size` | the 4 MB default, for incoming **requests** | +| `spring.grpc.server.inbound.metadata.max-size` | header size limit | +| `spring.grpc.server.shutdown.grace-period` | | +| `spring.grpc.server.keepalive.time` | | +| `spring.grpc.server.keepalive.timeout` | | +| `spring.grpc.server.keepalive.permit.time` | minimum client ping interval the server tolerates | +| `spring.grpc.server.keepalive.permit.without-calls` | | +| `spring.grpc.server.keepalive.connection.max-age` | | +| `spring.grpc.server.keepalive.connection.max-idle-time` | | +| `spring.grpc.server.keepalive.connection.grace-period` | | +| `spring.grpc.server.ssl.enabled` / `.bundle` / `.client-auth` / `.secure` | uses Boot SSL bundles | +| `spring.grpc.server.health.enabled` | standard gRPC health service | +| `spring.grpc.server.health.service` | | +| `spring.grpc.server.health.include-overall-health` | | +| `spring.grpc.server.health.status.mapping` / `.order` | actuator status → gRPC serving status | +| `spring.grpc.server.health.schedule.enabled` / `.delay` / `.period` | | +| `spring.grpc.server.health.services.validate-membership` | | +| `spring.grpc.server.reflection.enabled` | needed for `grpcurl` without local `.proto` files | +| `spring.grpc.server.observation.enabled` | Micrometer observations | +| `spring.grpc.server.netty.transport` | | +| `spring.grpc.server.netty.domain-socket-path` | | +| `spring.grpc.server.servlet.enabled` / `.validate-http2` | gRPC over the servlet container | +| `spring.grpc.server.security.csrf.enabled` | | +| `spring.grpc.server.factory.enabled` | | + +## 3. Client properties — exact names + +The client side is a **map keyed by channel name**: + +``` +spring.grpc.client.channel.. +``` + +**`channel`, singular.** `channels` is wrong and produces no error — see below. + +| Property | Notes | +|---|---| +| `spring.grpc.client.channel..target` | **`target`, not `address`** | +| `spring.grpc.client.channel..user-agent` | | +| `spring.grpc.client.channel..bypass-certificate-validation` | test/dev only | +| `spring.grpc.client.channel..default.deadline` | **the default deadline gRPC otherwise lacks** | +| `spring.grpc.client.channel..default.load-balancing-policy` | e.g. `round_robin` (default is `pick_first`) | +| `spring.grpc.client.channel..inbound.message.max-size` | 4 MB default, for incoming **responses** | +| `spring.grpc.client.channel..inbound.metadata.max-size` | | +| `spring.grpc.client.channel..keepalive.time` / `.timeout` / `.without-calls` | | +| `spring.grpc.client.channel..idle.timeout` | | +| `spring.grpc.client.channel..ssl.enabled` / `.bundle` | | +| `spring.grpc.client.channel..health.enabled` / `.service-name` | client-side health checking | +| `spring.grpc.client.channel..service-config.*` | retry, load balancing, throttling | +| `spring.grpc.client.enabled` | | +| `spring.grpc.client.inprocess.enabled` | register the in-process channel factory | +| `spring.grpc.client.observation.enabled` | | +| `spring.grpc.client.channelfactory.enabled` | | + +`service-config` maps to the gRPC service config record with `loadbalancing`, `method`, +`retrythrottling` and `healthcheck` sections. + +## 4. The two mistakes, and their symptoms + +| Wrong | Right | Symptom | +|---|---|---| +| `spring.grpc.server.in-process.name` | `spring.grpc.server.inprocess.name` | property silently ignored; server never binds in-process | +| `spring.grpc.client.channels.x.address` | `spring.grpc.client.channel.x.target` | `UNAVAILABLE: Unable to resolve host x` / `UnknownHostException: x` | + +The second is the expensive one. An unmatched channel name is handed to the DNS resolver **as a +literal target**, so the error names your logical channel as if it were a hostname — and sends you +to investigate DNS, service discovery and networking instead of a typo. Both cost real time while +building this repository. + +**Guard against it:** add `spring-boot-configuration-processor` and your IDE will flag unknown +`spring.grpc.*` keys. It will not catch a wrong value inside the `channel.` map (the keys +there are arbitrary), but it catches everything else. diff --git a/pom.xml b/pom.xml new file mode 100644 index 0000000..22a2413 --- /dev/null +++ b/pom.xml @@ -0,0 +1,108 @@ + + + + 4.0.0 + + + org.springframework.boot + spring-boot-starter-parent + 4.1.0 + + + + com.ankurm.grpc + spring-grpc-boot4 + 1.0.0 + Spring gRPC with Spring Boot 4 + + + 25 + UTF-8 + + + + + + org.springframework.boot + spring-boot-starter-grpc-server + + + org.springframework.boot + spring-boot-starter-grpc-client + + + + org.springframework.boot + spring-boot-starter-test + test + + + org.springframework.boot + spring-boot-starter-grpc-server-test + test + + + org.springframework.boot + spring-boot-starter-grpc-client-test + test + + + org.awaitility + awaitility + test + + + + + + + + io.github.ascopes + protobuf-maven-plugin + + ${protobuf-java.version} + + + io.grpc + protoc-gen-grpc-java + ${grpc-java.version} + + + + + + + generate + + + + + + + org.apache.maven.plugins + maven-surefire-plugin + + false + false + + + + + diff --git a/results-full.txt b/results-full.txt new file mode 100644 index 0000000..cbd164d Binary files /dev/null and b/results-full.txt differ diff --git a/src/main/java/com/ankurm/grpc/Application.java b/src/main/java/com/ankurm/grpc/Application.java new file mode 100644 index 0000000..92283c4 --- /dev/null +++ b/src/main/java/com/ankurm/grpc/Application.java @@ -0,0 +1,19 @@ +package com.ankurm.grpc; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; + +/** + * Companion application for + * Spring gRPC with Spring Boot 4. + * + *

The service lives in {@code orders/}; the interesting material is in {@code src/test}, where + * each failure mode is reproduced and then fixed. + */ +@SpringBootApplication +public class Application { + + public static void main(String[] args) { + SpringApplication.run(Application.class, args); + } +} diff --git a/src/main/java/com/ankurm/grpc/orders/OrderExceptionAdvice.java b/src/main/java/com/ankurm/grpc/orders/OrderExceptionAdvice.java new file mode 100644 index 0000000..f44d761 --- /dev/null +++ b/src/main/java/com/ankurm/grpc/orders/OrderExceptionAdvice.java @@ -0,0 +1,87 @@ +package com.ankurm.grpc.orders; + +import io.grpc.Metadata; +import io.grpc.Status; +import io.grpc.StatusException; +import org.springframework.grpc.server.advice.GrpcAdvice; +import org.springframework.grpc.server.advice.GrpcExceptionHandler; + +/** + * Centralised exception mapping -- the gRPC equivalent of {@code @RestControllerAdvice}. + * + *

Why this matters more in gRPC than in REST

+ * An exception that escapes a gRPC handler becomes {@code UNKNOWN} with a null description. + * Not a 500 with a body you can read: {@code UNKNOWN}, and nothing else. gRPC deliberately refuses + * to serialise your exception message across a service boundary, because doing so is an + * information-disclosure risk. + * + *

So without mapping, every domain failure looks identical to every bug, and callers cannot tell + * "this order does not exist" (never retry) from "the database is down" (retry). Retry policies, + * circuit breakers and dashboards all key off the status code, so an unmapped exception does not + * merely lose information -- it makes every downstream behaviour wrong. + * + *

The two mechanisms

+ * Spring gRPC offers both, and they compose: + * + *
    + *
  1. Annotation-based (this class): a {@code @GrpcAdvice} bean with + * {@code @GrpcExceptionHandler} methods, one per exception type. Familiar, readable, and the + * right default.
    2. + *
  2. Functional: a bean implementing + * {@code org.springframework.grpc.server.exception.GrpcExceptionHandler}, a single method + * {@code StatusException handleException(Throwable)}. Better for a catch-all, or for mapping + * driven by data rather than by type.
    3. + *
+ * + *

Note the two interfaces share the simple name {@code GrpcExceptionHandler} in different + * packages ({@code ...server.advice} for the annotation, {@code ...server.exception} for the + * functional interface). An IDE will happily import the wrong one and the failure is a confusing + * "annotation type not applicable" or "is not a functional interface". + * + *

Returning trailers

+ * A handler returns {@link StatusException}, which can carry {@link Metadata}. Put machine-readable + * context there rather than formatting it into the description string: descriptions are for humans + * reading logs, trailers are for code making decisions. + */ +@GrpcAdvice +public class OrderExceptionAdvice { + + /** Domain "not found" becomes NOT_FOUND, with the offending id in a trailer. */ + @GrpcExceptionHandler(OrderNotFoundException.class) + public StatusException handleNotFound(OrderNotFoundException ex) { + Metadata trailers = new Metadata(); + trailers.put(OrderServiceImpl.REASON_KEY, "ORDER_NOT_FOUND"); + return Status.NOT_FOUND + .withDescription(ex.getMessage()) + .asException(trailers); + } + + /** + * Bad input becomes INVALID_ARGUMENT. + * + *

Both of these are in the "never retry" family, which is the useful half of the + * distinction: a client that retries a NOT_FOUND is wasting capacity on an answer that will + * not change. + */ + @GrpcExceptionHandler(IllegalArgumentException.class) + public StatusException handleBadInput(IllegalArgumentException ex) { + return Status.INVALID_ARGUMENT + .withDescription(ex.getMessage()) + .asException(); + } + + /** + * A catch-all, so that an unexpected bug still produces a deliberate INTERNAL rather + * than a bare UNKNOWN. + * + *

Deliberately does not include {@code ex.getMessage()} in the description: an + * unexpected exception's message may contain SQL, file paths or user data, and this crosses a + * service boundary. Log it server-side; send the caller a correlation id instead. + */ + @GrpcExceptionHandler(Exception.class) + public StatusException handleEverythingElse(Exception ex) { + return Status.INTERNAL + .withDescription("internal error") + .asException(); + } +} diff --git a/src/main/java/com/ankurm/grpc/orders/OrderNotFoundException.java b/src/main/java/com/ankurm/grpc/orders/OrderNotFoundException.java new file mode 100644 index 0000000..89e54fa --- /dev/null +++ b/src/main/java/com/ankurm/grpc/orders/OrderNotFoundException.java @@ -0,0 +1,23 @@ +package com.ankurm.grpc.orders; + +/** + * An ordinary domain exception. It knows nothing about gRPC -- no {@code Status}, no + * {@code StatusRuntimeException}, no transport concepts at all. + * + *

That is the point. Mapping it to a gRPC status is the job of + * {@link OrderExceptionAdvice}, exactly as mapping a domain exception to an HTTP status is the + * job of a {@code @RestControllerAdvice} rather than of the service that throws it. + */ +public class OrderNotFoundException extends RuntimeException { + + private final String orderId; + + public OrderNotFoundException(String orderId) { + super("no order with id " + orderId); + this.orderId = orderId; + } + + public String orderId() { + return orderId; + } +} diff --git a/src/main/java/com/ankurm/grpc/orders/OrderServiceImpl.java b/src/main/java/com/ankurm/grpc/orders/OrderServiceImpl.java new file mode 100644 index 0000000..ca609ad --- /dev/null +++ b/src/main/java/com/ankurm/grpc/orders/OrderServiceImpl.java @@ -0,0 +1,231 @@ +package com.ankurm.grpc.orders; + +import com.google.protobuf.ByteString; +import io.grpc.Context; +import io.grpc.Metadata; +import io.grpc.Status; +import io.grpc.StatusRuntimeException; +import io.grpc.stub.StreamObserver; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.stereotype.Service; + +import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicLong; + +/** + * The service implementation. + * + *

On Spring Boot 4 a {@code @Service} bean that extends the generated {@code ImplBase} is + * discovered automatically: it is a {@link io.grpc.BindableService}, and + * {@code GrpcServerServicesAutoConfiguration} registers every such bean with the server. There is + * no {@code @GrpcService} annotation to apply and no registration code to write. + */ +@Service +public class OrderServiceImpl extends OrderServiceGrpc.OrderServiceImplBase { + + private static final Logger log = LoggerFactory.getLogger(OrderServiceImpl.class); + + /** Incremented whenever {@link #slowCall} is entered -- lets tests count server-side attempts. */ + private final AtomicInteger slowCallAttempts = new AtomicInteger(); + + /** Set when a server-streaming call notices that its client has gone away. */ + private final AtomicInteger cancellationsObserved = new AtomicInteger(); + + /** How far ListOrders got before being cancelled; proves the server stopped early. */ + private final AtomicInteger lastListProgress = new AtomicInteger(); + + /** + * Note what this method does not do: it throws plain domain exceptions and never + * mentions {@link Status}. {@link OrderExceptionAdvice} maps them, exactly as a + * {@code @RestControllerAdvice} maps exceptions to HTTP statuses. + */ + @Override + public void getOrder(GetOrderRequest request, StreamObserver responseObserver) { + if (request.getId().isBlank()) { + throw new IllegalArgumentException("order id must not be blank"); + } + if (request.getId().startsWith("missing-")) { + throw new OrderNotFoundException(request.getId()); + } + if (request.getId().equals("boom")) { + // An unexpected bug, to show what the catch-all handler does with it. + throw new IllegalStateException("simulated internal failure with sensitive detail"); + } + responseObserver.onNext(order(request.getId())); + responseObserver.onCompleted(); + } + + /** + * Server streaming, written the way it should be: checking for cancellation between + * messages. + * + *

If the client goes away -- it cancelled, it timed out, its process died -- gRPC does not + * interrupt your thread. It sets a flag on the {@link Context}. A server that never checks that + * flag keeps computing, keeps querying the database, and keeps calling {@code onNext} into a + * closed stream for the full duration of the work. On a long stream that is a real and + * surprisingly common source of wasted capacity. + */ + @Override + public void listOrders(ListOrdersRequest request, StreamObserver responseObserver) { + int count = request.getCount(); + for (int i = 0; i < count; i++) { + if (Context.current().isCancelled()) { + cancellationsObserved.incrementAndGet(); + lastListProgress.set(i); + log.info("client cancelled after {} of {} messages -- stopping work", i, count); + // Do NOT call onCompleted/onError here: the stream is already closed. Just return. + return; + } + responseObserver.onNext(order("order-" + i)); + if (request.getDelayMillis() > 0) { + sleep(request.getDelayMillis()); + } + } + lastListProgress.set(count); + responseObserver.onCompleted(); + } + + /** + * Client streaming. The returned observer receives the client's messages; the response is sent + * exactly once, from {@code onCompleted}. + */ + @Override + public StreamObserver submitOrders(StreamObserver responseObserver) { + AtomicInteger accepted = new AtomicInteger(); + AtomicLong total = new AtomicLong(); + + return new StreamObserver<>() { + @Override + public void onNext(Order value) { + accepted.incrementAndGet(); + total.addAndGet(value.getAmountCents()); + } + + @Override + public void onError(Throwable t) { + // The CLIENT failed or cancelled. The stream is already dead -- responding here + // throws. Log and release resources, nothing else. + log.info("client-streaming call failed: {}", t.toString()); + } + + @Override + public void onCompleted() { + responseObserver.onNext(SubmitSummary.newBuilder() + .setAccepted(accepted.get()) + .setTotalCents(total.get()) + .build()); + responseObserver.onCompleted(); + } + }; + } + + /** + * Bidirectional streaming. + * + *

The thread-safety rule that is easy to miss: a {@link StreamObserver} is not + * thread-safe. In a bidi call it is legal to call {@code onNext} on the response observer from + * a different thread than the one delivering requests -- but only if you serialise those calls + * yourself. Here every response is emitted from inside {@code onNext}, on the delivering + * thread, which is the simplest correct arrangement. + */ + @Override + public StreamObserver sync(StreamObserver responseObserver) { + return new StreamObserver<>() { + @Override + public void onNext(Order value) { + responseObserver.onNext(value.toBuilder().setStatus(Order.Status.PAID).build()); + } + + @Override + public void onError(Throwable t) { + log.info("bidi call failed: {}", t.toString()); + } + + @Override + public void onCompleted() { + responseObserver.onCompleted(); + } + }; + } + + /** Returns a payload of exactly the requested size, for exercising message size limits. */ + @Override + public void getLargePayload(SizeRequest request, StreamObserver responseObserver) { + byte[] data = new byte[Math.max(0, request.getSizeBytes())]; + responseObserver.onNext(Payload.newBuilder().setData(ByteString.copyFrom(data)).build()); + responseObserver.onCompleted(); + } + + /** Sleeps, so deadlines and cancellation can be exercised deterministically. */ + @Override + public void slowCall(SlowRequest request, StreamObserver responseObserver) { + slowCallAttempts.incrementAndGet(); + sleep(request.getSleepMillis()); + if (Context.current().isCancelled()) { + // The deadline already expired. The client has long since given up; sending now throws + // and adds nothing but noise to the logs. + log.info("slowCall finished but the context was already cancelled -- not responding"); + return; + } + responseObserver.onNext(order("slow")); + responseObserver.onCompleted(); + } + + /** + * Always fails, with a machine-readable reason attached as trailing metadata. + * + *

Returning a bare {@code Status.INTERNAL} tells a caller nothing they can act on. Attaching + * trailers -- or, better, a {@code google.rpc.Status} with typed detail messages -- is how gRPC + * carries structured errors. This uses plain metadata to keep the proto dependency-free. + */ + @Override + public void alwaysFails(GetOrderRequest request, StreamObserver responseObserver) { + Metadata trailers = new Metadata(); + trailers.put(REASON_KEY, "ORDER_LOCKED"); + trailers.put(RETRY_AFTER_KEY, "30"); + responseObserver.onError(new StatusRuntimeException( + Status.FAILED_PRECONDITION.withDescription("order " + request.getId() + " is locked"), + trailers)); + } + + public static final Metadata.Key REASON_KEY = + Metadata.Key.of("x-failure-reason", Metadata.ASCII_STRING_MARSHALLER); + public static final Metadata.Key RETRY_AFTER_KEY = + Metadata.Key.of("x-retry-after-seconds", Metadata.ASCII_STRING_MARSHALLER); + + private static Order order(String id) { + return Order.newBuilder() + .setId(id) + .setCustomer("ankur") + .setAmountCents(1_999) + .setStatus(Order.Status.PENDING) + .build(); + } + + private static void sleep(long millis) { + try { + Thread.sleep(millis); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + } + } + + public int slowCallAttempts() { + return slowCallAttempts.get(); + } + + public int cancellationsObserved() { + return cancellationsObserved.get(); + } + + public int lastListProgress() { + return lastListProgress.get(); + } + + public void reset() { + slowCallAttempts.set(0); + cancellationsObserved.set(0); + lastListProgress.set(0); + } +} diff --git a/src/main/proto/orders.proto b/src/main/proto/orders.proto new file mode 100644 index 0000000..2aec69b --- /dev/null +++ b/src/main/proto/orders.proto @@ -0,0 +1,80 @@ +syntax = "proto3"; + +package com.ankurm.grpc.orders; + +option java_multiple_files = true; +option java_package = "com.ankurm.grpc.orders"; +option java_outer_classname = "OrdersProto"; + +// A deliberately small service that still exercises all four gRPC call types, because the +// interesting failure modes differ sharply between them. +service OrderService { + + // Unary: one request, one response. The 90% case, and the only one most tutorials cover. + rpc GetOrder (GetOrderRequest) returns (Order); + + // Server streaming: one request, many responses. Where deadlines and cancellation start to + // matter, and where "the client went away" becomes something you have to handle. + rpc ListOrders (ListOrdersRequest) returns (stream Order); + + // Client streaming: many requests, one response. Where flow control and half-close appear. + rpc SubmitOrders (stream Order) returns (SubmitSummary); + + // Bidirectional streaming: the one that will teach you about StreamObserver thread safety. + rpc Sync (stream Order) returns (stream Order); + + // Used by the troubleshooting suite to return payloads of a requested size, so that message + // size limits can be demonstrated rather than described. + rpc GetLargePayload (SizeRequest) returns (Payload); + + // Sleeps for a requested duration so that deadlines, cancellation and keepalive behaviour can + // be exercised deterministically. + rpc SlowCall (SlowRequest) returns (Order); + + // Always fails, with rich error details attached. + rpc AlwaysFails (GetOrderRequest) returns (Order); +} + +message GetOrderRequest { + string id = 1; +} + +message ListOrdersRequest { + int32 count = 1; + // Milliseconds to wait between emitted messages; lets tests exercise slow producers. + int32 delay_millis = 2; +} + +message Order { + string id = 1; + string customer = 2; + int64 amount_cents = 3; + Status status = 4; + + enum Status { + // proto3 requires the zero value to be the "unset" case. Naming it UNSPECIFIED rather than + // reusing a real state is a convention worth keeping: it makes "field absent" distinguishable + // from "field genuinely has the first value", which is otherwise impossible in proto3. + STATUS_UNSPECIFIED = 0; + PENDING = 1; + PAID = 2; + CANCELLED = 3; + } +} + +message SubmitSummary { + int32 accepted = 1; + int64 total_cents = 2; +} + +message SizeRequest { + int32 size_bytes = 1; +} + +message Payload { + bytes data = 1; +} + +message SlowRequest { + int32 sleep_millis = 1; +} diff --git a/src/test/java/com/ankurm/grpc/_01_basics/FourCallTypesTest.java b/src/test/java/com/ankurm/grpc/_01_basics/FourCallTypesTest.java new file mode 100644 index 0000000..0b74cb1 --- /dev/null +++ b/src/test/java/com/ankurm/grpc/_01_basics/FourCallTypesTest.java @@ -0,0 +1,147 @@ +package com.ankurm.grpc._01_basics; + +import com.ankurm.grpc.orders.GetOrderRequest; +import com.ankurm.grpc.orders.ListOrdersRequest; +import com.ankurm.grpc.orders.Order; +import com.ankurm.grpc.orders.OrderServiceGrpc; +import com.ankurm.grpc.orders.SubmitSummary; +import com.ankurm.grpc.support.GrpcTestBase; +import com.ankurm.grpc.support.Report; +import io.grpc.stub.StreamObserver; +import org.junit.jupiter.api.Test; + +import java.util.ArrayList; +import java.util.Iterator; +import java.util.List; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicReference; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * All four gRPC call types against a Spring Boot 4 server, with the differences that matter. + * + *

Most introductions show unary and stop. The three streaming forms are where the interesting + * failure modes live, and they behave differently enough that experience with one does not transfer + * to the others. + */ +class FourCallTypesTest extends GrpcTestBase { + + @Test + void unary() { + Report.title("Unary: one request, one response"); + + Order order = blockingStub().getOrder(GetOrderRequest.newBuilder().setId("abc").build()); + + Report.bullet("id=%s customer=%s amountCents=%d status=%s", + order.getId(), order.getCustomer(), order.getAmountCents(), order.getStatus()); + assertThat(order.getId()).isEqualTo("abc"); + + Report.takeaway("A @Service extending the generated ImplBase is registered automatically:"); + Report.takeaway("it is a BindableService, and Boot 4 wires every such bean into the server."); + } + + @Test + void serverStreaming() { + Report.title("Server streaming: one request, many responses"); + + Iterator it = blockingStub() + .listOrders(ListOrdersRequest.newBuilder().setCount(5).build()); + + List ids = new ArrayList<>(); + it.forEachRemaining(o -> ids.add(o.getId())); + + Report.bullet("received %d messages: %s", ids.size(), ids); + assertThat(ids).hasSize(5); + + Report.takeaway("The blocking stub returns an Iterator. Each next() may block, and an"); + Report.takeaway("exception surfaces mid-iteration -- so a try/catch around the loop body"); + Report.takeaway("is not the same as one around the call. Wrap the whole iteration."); + } + + @Test + void clientStreaming() throws Exception { + Report.title("Client streaming: many requests, one response"); + + AtomicReference summary = new AtomicReference<>(); + CountDownLatch done = new CountDownLatch(1); + + StreamObserver requests = asyncStub().submitOrders(new StreamObserver<>() { + @Override + public void onNext(SubmitSummary value) { + summary.set(value); + } + + @Override + public void onError(Throwable t) { + done.countDown(); + } + + @Override + public void onCompleted() { + done.countDown(); + } + }); + + for (int i = 0; i < 3; i++) { + requests.onNext(Order.newBuilder().setId("o" + i).setAmountCents(1000).build()); + } + // Half-close: "I have no more requests". Forgetting this is the single most common + // client-streaming bug -- the server's onCompleted never fires and the call hangs until + // the deadline, or forever if there is no deadline. + requests.onCompleted(); + + assertThat(done.await(10, TimeUnit.SECONDS)).isTrue(); + Report.bullet("accepted=%d totalCents=%d", + summary.get().getAccepted(), summary.get().getTotalCents()); + assertThat(summary.get().getAccepted()).isEqualTo(3); + + Report.takeaway("requests.onCompleted() is the half-close. Without it the call hangs until"); + Report.takeaway("the deadline -- and if you did not set a deadline, it hangs forever."); + } + + @Test + void bidirectionalStreaming() throws Exception { + Report.title("Bidirectional streaming: many requests, many responses"); + + List received = new ArrayList<>(); + CountDownLatch done = new CountDownLatch(1); + + StreamObserver requests = asyncStub().sync(new StreamObserver<>() { + @Override + public void onNext(Order value) { + received.add(value); + } + + @Override + public void onError(Throwable t) { + done.countDown(); + } + + @Override + public void onCompleted() { + done.countDown(); + } + }); + + for (int i = 0; i < 4; i++) { + requests.onNext(Order.newBuilder().setId("b" + i).setStatus(Order.Status.PENDING).build()); + } + requests.onCompleted(); + + assertThat(done.await(10, TimeUnit.SECONDS)).isTrue(); + Report.bullet("sent 4, received %d, all status=%s", + received.size(), received.isEmpty() ? "-" : received.getFirst().getStatus()); + assertThat(received).hasSize(4); + assertThat(received).allMatch(o -> o.getStatus() == Order.Status.PAID); + + Report.takeaway("Request and response streams are INDEPENDENT. The server may respond"); + Report.takeaway("before you finish sending, or not at all until you half-close. Do not"); + Report.takeaway("assume a request/response pairing -- that is your protocol's job, not gRPC's."); + Report.takeaway(""); + Report.takeaway("StreamObserver is NOT thread-safe. Calling onNext from two threads without"); + Report.takeaway("synchronisation corrupts the stream, and the symptom is usually a"); + Report.takeaway("deserialization error on the far side rather than anything pointing here."); + } +} diff --git a/src/test/java/com/ankurm/grpc/_02_troubleshooting/HardToDiagnoseTest.java b/src/test/java/com/ankurm/grpc/_02_troubleshooting/HardToDiagnoseTest.java new file mode 100644 index 0000000..2234431 --- /dev/null +++ b/src/test/java/com/ankurm/grpc/_02_troubleshooting/HardToDiagnoseTest.java @@ -0,0 +1,265 @@ +package com.ankurm.grpc._02_troubleshooting; + +import com.ankurm.grpc.orders.GetOrderRequest; +import com.ankurm.grpc.orders.ListOrdersRequest; +import com.ankurm.grpc.orders.Order; +import com.ankurm.grpc.orders.OrderServiceGrpc; +import com.ankurm.grpc.orders.OrderServiceImpl; +import com.ankurm.grpc.orders.Payload; +import com.ankurm.grpc.orders.SizeRequest; +import com.ankurm.grpc.orders.SlowRequest; +import com.ankurm.grpc.support.GrpcTestBase; +import com.ankurm.grpc.support.Report; +import io.grpc.Metadata; +import io.grpc.Status; +import io.grpc.StatusRuntimeException; +import io.grpc.stub.StreamObserver; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; + +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicReference; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; +import static org.awaitility.Awaitility.await; + +/** + *

The failures that cost hours

+ * + *

Each test here reproduces a problem that is hard to diagnose from its symptom, then + * shows the fix. They are grouped in one class because they share a server and the suite is faster + * that way; each is independent. + * + *

Marked in the transcript as {@code [PROBLEM]} and {@code [FIX]}. + */ +class HardToDiagnoseTest extends GrpcTestBase { + + @Autowired + OrderServiceImpl service; + + @BeforeEach + void reset() { + service.reset(); + } + + // ======================================================================================= + // 1. The 4 MB message limit + // ======================================================================================= + + /** + * Symptom: {@code RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 4194304}. + * Works in dev, fails in production, or works for 99% of requests and fails for the big ones. + * + *

Why it is confusing: the limit is per message, it applies to the + * receiving side, and client and server are configured separately. So raising + * it on the server does nothing for a large response, which is the client's inbound + * limit. Half the reports of "I raised the limit and it still fails" are this asymmetry. + */ + @Test + void inProcessTransportDoesNotEnforceMessageSizeLimits() { + Report.title("1. The 4 MB limit -- and why your integration tests will not catch it"); + + int fourMb = 4 * 1024 * 1024; + + Report.section("Sending a 4 MB + 1 KB response over the IN-PROCESS transport"); + Payload big = blockingStub().getLargePayload( + SizeRequest.newBuilder().setSizeBytes(fourMb + 1024).build()); + Report.bullet("received %,d bytes -- no error", big.getData().size()); + + assertThat(big.getData().size()) + .as("in-process transport happily delivers a message over the default limit") + .isEqualTo(fourMb + 1024); + + Report.problem("This message is OVER the 4 MB default limit and nothing complained."); + Report.problem("The in-process transport passes message objects BY REFERENCE -- it never"); + Report.problem("serialises them -- so there are no bytes to measure and the limit cannot"); + Report.problem("be applied. Over a real Netty transport the same call fails with:"); + Report.problem(" RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 4194304"); + + Report.takeaway("This is a genuinely nasty testing trap. In-process transport is otherwise"); + Report.takeaway("an excellent test transport -- interceptors, statuses, deadlines and"); + Report.takeaway("cancellation all behave correctly -- but message size limits, compression"); + Report.takeaway("and anything else that depends on the wire format do NOT apply."); + Report.takeaway("A payload-size regression will pass every in-process test you have."); + Report.takeaway(""); + Report.takeaway("Test size limits against a real port, or not at all -- but do not believe"); + Report.takeaway("a green in-process suite on this point."); + + Report.section("The fix in production, for when you do hit it"); + Report.fix("The limit applies to the RECEIVER, and the two sides are configured separately:"); + Report.fix(" large RESPONSE -> client inbound limit"); + Report.fix(" spring.grpc.client.channel..inbound.message.max-size=16MB"); + Report.fix(" large REQUEST -> server inbound limit"); + Report.fix(" spring.grpc.server.inbound.message.max-size=16MB"); + Report.fix("Per call, without touching configuration:"); + Report.fix(" stub.withMaxInboundMessageSize(16 * 1024 * 1024)"); + Report.fix(""); + Report.fix("Half of all 'I raised the limit and it still fails' reports are this"); + Report.fix("asymmetry: raising it on the server does nothing for a large RESPONSE."); + Report.fix(""); + Report.fix("Do not raise it globally. A large limit turns a malformed request into an OOM."); + Report.fix("Prefer streaming: the limit is per MESSAGE, so 1000 small messages are fine."); + } + + // ======================================================================================= + // 2. No deadline + // ======================================================================================= + + /** + * Symptom: a thread pool fills up and the service stops responding, with no errors in + * the logs. Or: a retry storm during a downstream slowdown. + * + *

Why it is confusing: gRPC calls have no default deadline. A call with no + * deadline waits forever. Nothing warns you, and in testing the server always responds quickly, + * so it never surfaces until the day something downstream is slow. + */ + @Test + void callsHaveNoDefaultDeadline() { + Report.title("2. gRPC has NO default deadline -- a hung server hangs you forever"); + + Report.section("A call with an explicit deadline shorter than the server takes"); + long start = System.nanoTime(); + assertThatThrownBy(() -> blockingStub() + .withDeadlineAfter(300, TimeUnit.MILLISECONDS) + .slowCall(SlowRequest.newBuilder().setSleepMillis(3_000).build())) + .isInstanceOf(StatusRuntimeException.class) + .satisfies(e -> { + Status s = ((StatusRuntimeException) e).getStatus(); + Report.bullet("%s after %d ms", s.getCode(), + (System.nanoTime() - start) / 1_000_000); + assertThat(s.getCode()).isEqualTo(Status.Code.DEADLINE_EXCEEDED); + }); + + Report.problem("Without .withDeadlineAfter(...) that call would have blocked for the full"); + Report.problem("3 seconds -- and if the server never responded, forever."); + + Report.section("The fix"); + Report.fix("Per call: stub.withDeadlineAfter(2, TimeUnit.SECONDS)"); + Report.fix("Per channel: register a DefaultDeadlineSetupClientInterceptor, or set"); + Report.fix(" spring.grpc.client.channel..default.deadline=2s"); + Report.fix("Treat a missing deadline as a code-review failure, like a missing timeout"); + Report.fix("on an HTTP client."); + + Report.takeaway("A deadline is ABSOLUTE and propagates: if service A calls B with 2s"); + Report.takeaway("remaining, B sees 2s, not a fresh 2s. That is the property that stops a"); + Report.takeaway("deep call chain from multiplying its timeouts -- and the reason you should"); + Report.takeaway("set the deadline at the EDGE, not re-set it at every hop."); + } + + // ======================================================================================= + // 3. Cancellation the server never notices + // ======================================================================================= + + /** + * Symptom: the client timed out ten minutes ago, and the server is still burning CPU and + * database connections on its request. + * + *

Why it is confusing: gRPC does not interrupt your thread when a call is cancelled. + * It flips a flag on the {@link io.grpc.Context}. A server that never checks it keeps working. + */ + @Test + void serverMustCheckForCancellationItself() throws Exception { + Report.title("3. Cancellation does not interrupt your thread -- you must check for it"); + + Report.section("Client starts a 200-message stream, then cancels after a few messages"); + CountDownLatch got3 = new CountDownLatch(3); + AtomicReference clientError = new AtomicReference<>(); + + io.grpc.stub.ClientCallStreamObserver requestStream = + new io.grpc.stub.ClientCallStreamObserver<>() { + @Override public boolean isReady() { return true; } + @Override public void setOnReadyHandler(Runnable r) { } + @Override public void disableAutoInboundFlowControl() { } + @Override public void request(int count) { } + @Override public void setMessageCompression(boolean enable) { } + @Override public void cancel(String message, Throwable cause) { } + @Override public void onNext(ListOrdersRequest value) { } + @Override public void onError(Throwable t) { } + @Override public void onCompleted() { } + }; + + io.grpc.Context.CancellableContext ctx = io.grpc.Context.current().withCancellation(); + ctx.run(() -> asyncStub().listOrders( + ListOrdersRequest.newBuilder().setCount(200).setDelayMillis(20).build(), + new StreamObserver<>() { + @Override public void onNext(Order value) { got3.countDown(); } + @Override public void onError(Throwable t) { clientError.set(t); } + @Override public void onCompleted() { } + })); + + assertThat(got3.await(10, TimeUnit.SECONDS)).as("stream started").isTrue(); + Report.bullet("received 3 messages, now cancelling"); + ctx.cancel(new RuntimeException("client gave up")); + + await().atMost(15, TimeUnit.SECONDS) + .until(() -> service.cancellationsObserved() > 0); + + Report.bullet("server noticed cancellation after %d of 200 messages", service.lastListProgress()); + assertThat(service.lastListProgress()).isLessThan(200); + + Report.fix("The server loop checks Context.current().isCancelled() between messages and"); + Report.fix("returns. Without that check it would have produced all 200 messages into a"); + Report.fix("dead stream, doing every database read and every serialization for nothing."); + + Report.takeaway("Any server handler that loops, or that does work in stages, should check"); + Report.takeaway("Context.current().isCancelled(). For blocking work, propagate the Context"); + Report.takeaway("to worker threads with Context.current().wrap(runnable) -- otherwise the"); + Report.takeaway("cancellation flag is invisible to them."); + Report.takeaway(""); + Report.takeaway("Also: after cancellation the stream is CLOSED. Calling onNext/onCompleted"); + Report.takeaway("on it throws IllegalStateException. Just return."); + } + + // ======================================================================================= + // 4. Errors that tell the caller nothing + // ======================================================================================= + + /** + * Symptom: {@code UNKNOWN} statuses everywhere, and callers that cannot tell a retryable + * failure from a permanent one. + * + *

Why it is confusing: any exception that escapes a gRPC handler becomes + * {@code UNKNOWN} with no message, because leaking exception text across a service + * boundary would be an information disclosure risk. So the useful part of your error is + * discarded by default. + */ + @Test + void statusCodesAndTrailersCarryTheActualReason() { + Report.title("4. Errors: UNKNOWN by default, useful only if you make them so"); + + Report.section("A handler that fails with a proper status and trailing metadata"); + assertThatThrownBy(() -> blockingStub() + .alwaysFails(GetOrderRequest.newBuilder().setId("o-42").build())) + .isInstanceOf(StatusRuntimeException.class) + .satisfies(e -> { + StatusRuntimeException sre = (StatusRuntimeException) e; + Report.bullet("code : %s", sre.getStatus().getCode()); + Report.bullet("description : %s", sre.getStatus().getDescription()); + + Metadata trailers = sre.getTrailers(); + String reason = trailers == null ? null : trailers.get(OrderServiceImpl.REASON_KEY); + String retryAfter = trailers == null ? null : trailers.get(OrderServiceImpl.RETRY_AFTER_KEY); + Report.bullet("x-failure-reason : %s", reason); + Report.bullet("x-retry-after-seconds : %s", retryAfter); + + assertThat(sre.getStatus().getCode()).isEqualTo(Status.Code.FAILED_PRECONDITION); + assertThat(reason).isEqualTo("ORDER_LOCKED"); + }); + + Report.problem("If the handler had simply thrown IllegalStateException, the caller would"); + Report.problem("have received UNKNOWN with a null description. Nothing actionable at all."); + + Report.fix("Throw StatusRuntimeException with a code that means something:"); + Report.fix(" NOT_FOUND / INVALID_ARGUMENT / FAILED_PRECONDITION -- do NOT retry"); + Report.fix(" UNAVAILABLE / DEADLINE_EXCEEDED / RESOURCE_EXHAUSTED -- retry may help"); + Report.fix("Attach machine-readable context as trailers, not in the description string."); + + Report.takeaway("The status code is the contract. Clients, retry policies, circuit breakers"); + Report.takeaway("and dashboards all key off it, so choosing it carelessly makes every"); + Report.takeaway("downstream behaviour wrong -- most damagingly, it makes non-retryable"); + Report.takeaway("failures look retryable and turns one bad request into a storm."); + } +} diff --git a/src/test/java/com/ankurm/grpc/_03_exceptions/ExceptionMappingTest.java b/src/test/java/com/ankurm/grpc/_03_exceptions/ExceptionMappingTest.java new file mode 100644 index 0000000..757a7de --- /dev/null +++ b/src/test/java/com/ankurm/grpc/_03_exceptions/ExceptionMappingTest.java @@ -0,0 +1,97 @@ +package com.ankurm.grpc._03_exceptions; + +import com.ankurm.grpc.orders.GetOrderRequest; +import com.ankurm.grpc.orders.OrderServiceImpl; +import com.ankurm.grpc.support.GrpcTestBase; +import com.ankurm.grpc.support.Report; +import io.grpc.Metadata; +import io.grpc.Status; +import io.grpc.StatusRuntimeException; +import org.junit.jupiter.api.Test; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +/** + * Spring-style exception mapping for gRPC: {@code @GrpcAdvice} plus {@code @GrpcExceptionHandler}, + * the direct analogue of {@code @RestControllerAdvice} plus {@code @ExceptionHandler}. + * + *

The service throws plain domain exceptions and never mentions {@code Status}. The advice + * translates. These tests show the three cases that matter: a mapped domain exception, a mapped + * validation failure, and an unanticipated bug caught by the catch-all. + */ +class ExceptionMappingTest extends GrpcTestBase { + + @Test + void domainExceptionBecomesNotFoundWithTrailers() { + Report.title("Exception mapping: a domain exception becomes NOT_FOUND"); + + assertThatThrownBy(() -> blockingStub() + .getOrder(GetOrderRequest.newBuilder().setId("missing-42").build())) + .isInstanceOf(StatusRuntimeException.class) + .satisfies(e -> { + StatusRuntimeException sre = (StatusRuntimeException) e; + Metadata trailers = sre.getTrailers(); + Report.bullet("code : %s", sre.getStatus().getCode()); + Report.bullet("description : %s", sre.getStatus().getDescription()); + Report.bullet("x-failure-reason : %s", + trailers == null ? null : trailers.get(OrderServiceImpl.REASON_KEY)); + + assertThat(sre.getStatus().getCode()).isEqualTo(Status.Code.NOT_FOUND); + assertThat(sre.getStatus().getDescription()).contains("missing-42"); + assertThat(trailers).isNotNull(); + assertThat(trailers.get(OrderServiceImpl.REASON_KEY)).isEqualTo("ORDER_NOT_FOUND"); + }); + + Report.fix("The service method just did: throw new OrderNotFoundException(id);"); + Report.fix("@GrpcAdvice + @GrpcExceptionHandler(OrderNotFoundException.class) did the rest."); + } + + @Test + void validationFailureBecomesInvalidArgument() { + Report.title("Exception mapping: IllegalArgumentException becomes INVALID_ARGUMENT"); + + assertThatThrownBy(() -> blockingStub() + .getOrder(GetOrderRequest.newBuilder().setId("").build())) + .isInstanceOf(StatusRuntimeException.class) + .satisfies(e -> { + Status s = ((StatusRuntimeException) e).getStatus(); + Report.bullet("code : %s", s.getCode()); + Report.bullet("description : %s", s.getDescription()); + assertThat(s.getCode()).isEqualTo(Status.Code.INVALID_ARGUMENT); + }); + + Report.takeaway("NOT_FOUND and INVALID_ARGUMENT are both in the 'never retry' family."); + Report.takeaway("A client that retries either is burning capacity on an answer that"); + Report.takeaway("will not change. That distinction is the whole reason to map."); + } + + @Test + void unexpectedExceptionBecomesInternalWithoutLeakingDetail() { + Report.title("Exception mapping: the catch-all, and what it deliberately hides"); + + assertThatThrownBy(() -> blockingStub() + .getOrder(GetOrderRequest.newBuilder().setId("boom").build())) + .isInstanceOf(StatusRuntimeException.class) + .satisfies(e -> { + Status s = ((StatusRuntimeException) e).getStatus(); + Report.bullet("code : %s", s.getCode()); + Report.bullet("description : %s", s.getDescription()); + + assertThat(s.getCode()).isEqualTo(Status.Code.INTERNAL); + assertThat(s.getDescription()).isEqualTo("internal error"); + assertThat(s.getDescription()) + .as("the original message must not cross the service boundary") + .doesNotContain("sensitive detail"); + }); + + Report.problem("Without the advice this would have been UNKNOWN with a NULL description --"); + Report.problem("no code to branch on and no message to read."); + Report.fix("With it, callers get a deliberate INTERNAL, and the sensitive text stays"); + Report.fix("server-side where it belongs. Log it with a correlation id and return that."); + + Report.takeaway("Order matters: @GrpcExceptionHandler(Exception.class) is a catch-all, so"); + Report.takeaway("more specific handlers must exist for the types you care about -- they are"); + Report.takeaway("matched most-specific-first, exactly like @ExceptionHandler."); + } +} diff --git a/src/test/java/com/ankurm/grpc/support/GrpcTestBase.java b/src/test/java/com/ankurm/grpc/support/GrpcTestBase.java new file mode 100644 index 0000000..cdc6a28 --- /dev/null +++ b/src/test/java/com/ankurm/grpc/support/GrpcTestBase.java @@ -0,0 +1,95 @@ +package com.ankurm.grpc.support; + +import com.ankurm.grpc.Application; +import com.ankurm.grpc.orders.OrderServiceGrpc; +import io.grpc.ManagedChannel; +import org.junit.jupiter.api.AfterEach; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.grpc.client.GrpcChannelFactory; +import org.springframework.test.context.TestPropertySource; + +import java.util.ArrayList; +import java.util.List; + +/** + * Base class for every demonstration: boots the application with gRPC on the in-process + * transport and hands out stubs built on channels the test controls. + * + *

Why in-process rather than a real port

+ * The in-process transport is a real gRPC stack -- interceptors, metadata, statuses, deadlines, + * flow control and cancellation all behave normally -- but it skips sockets and TLS. That makes + * tests fast and free of port conflicts, and it is the right default for demonstrating protocol + * behaviour. + * + *

What it does NOT reproduce, and therefore what these tests deliberately cannot show: + * anything about the network. Keepalives, GOAWAY frames, idle-timeout disconnects, load-balancer + * behaviour, TLS negotiation and DNS re-resolution all need a socket. Those are documented in + * {@code docs/06-network-and-keepalive.md} rather than asserted here, because a test that claims to + * prove them over in-process transport would be lying. + * + *

Note also that message size limits are enforced by the in-process transport even though + * no bytes cross a socket, because the limit is applied at the message layer. So the size-limit + * demonstrations are genuine. + */ +@SpringBootTest(classes = Application.class) +@TestPropertySource(properties = { + // Bind gRPC to the in-process transport only. Without this Boot starts a Netty server on + // a real port, which works but makes the suite slower and flakier on CI. + // + // NOTE the exact property names -- they are easy to get wrong and the failure is opaque: + // spring.grpc.server.inprocess.name (NOT "in-process") + // spring.grpc.client.channel..target (SINGULAR "channel"; and "target", NOT "address") + // + // Getting the client one wrong produces no configuration error at all. An unmatched channel + // name is simply passed to the default name resolver as a DNS target, so you get + // UNAVAILABLE: Unable to resolve host orders + // Caused by: java.net.UnknownHostException: orders + // which sends you to look at networking, DNS and service discovery rather than at a typo in + // a property name. Both mistakes in this file cost real time while writing it. + "spring.grpc.server.inprocess.name=orders-test", + "spring.grpc.server.port=-1", + "spring.grpc.client.inprocess.enabled=true", + "spring.grpc.client.channel.orders.target=in-process:orders-test" +}) +public abstract class GrpcTestBase { + + @Autowired + protected GrpcChannelFactory channels; + + private final List opened = new ArrayList<>(); + + /** The in-process target this suite's server listens on. */ + protected static final String TARGET = "in-process:orders-test"; + + /** + * A channel to the test server, tracked so it is shut down after the test. + * + *

Note that this passes the full target rather than the logical channel name. + * {@code GrpcChannelFactory} is a composite: it asks each registered factory whether it + * {@code supports(target)} before any named-channel indirection is applied, so passing a bare + * name to {@code createChannel} in this configuration fails with + * {@code IllegalStateException: No grpc channel factory found that supports target : orders}. + * Named channels are still the right thing in application code, where the property-configured + * target is resolved for you -- see {@code NamedChannelTest}. + */ + protected ManagedChannel channel() { + ManagedChannel channel = channels.createChannel(TARGET); + opened.add(channel); + return channel; + } + + protected OrderServiceGrpc.OrderServiceBlockingStub blockingStub() { + return OrderServiceGrpc.newBlockingStub(channel()); + } + + protected OrderServiceGrpc.OrderServiceStub asyncStub() { + return OrderServiceGrpc.newStub(channel()); + } + + @AfterEach + void closeChannels() { + opened.forEach(ManagedChannel::shutdownNow); + opened.clear(); + } +} diff --git a/src/test/java/com/ankurm/grpc/support/Report.java b/src/test/java/com/ankurm/grpc/support/Report.java new file mode 100644 index 0000000..2649dc2 --- /dev/null +++ b/src/test/java/com/ankurm/grpc/support/Report.java @@ -0,0 +1,38 @@ +package com.ankurm.grpc.support; + +/** Formatting helper so every demonstration prints a readable, diffable transcript. */ +public final class Report { + + private Report() { + } + + public static void title(String text) { + System.out.println(); + System.out.println("=".repeat(78)); + System.out.println(text); + System.out.println("=".repeat(78)); + } + + public static void section(String text) { + System.out.println(); + System.out.println("-- " + text + " " + "-".repeat(Math.max(0, 74 - text.length()))); + } + + public static void bullet(String format, Object... args) { + System.out.printf(" " + format + "%n", args); + } + + /** A problem being reproduced. */ + public static void problem(String format, Object... args) { + System.out.printf(" [PROBLEM] " + format + "%n", args); + } + + /** The fix for the problem just reproduced. */ + public static void fix(String format, Object... args) { + System.out.printf(" [FIX] " + format + "%n", args); + } + + public static void takeaway(String format, Object... args) { + System.out.printf(">> " + format + "%n", args); + } +}