asmhatre/spring-boot-demo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Spring Boot 4 Actuator in production: endpoints, security, custom health indicators

Browse Source 
Companion repository for the ankurm.com article. Every transcript in docs/output/
was produced by running this project; scripts/run-all.sh regenerates all of them.

Verified against Spring Boot 4.1.1 / Framework 7.0.9 / Security 7.1.1 /
Micrometer 1.17.1 / kafka-clients 4.2.1 on Temurin JDK 25.0.4.1+1.
This commit is contained in:
Ankur Mhatre
2026-09-04 10:40:04 +05:30
commit 4b6cefa60a
64 changed files with 3195 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
docs/01-what-actuator-exposes.md Normal file
View File

@@ -0,0 +1,57 @@
[← README](../README.md) · **01 · What Actuator actually exposes** · [02 What changed in Spring Boot 4 →](02-boot-4-changes.md)
# 01 — What Actuator actually exposes
Add the starter, configure nothing, and ask the application what it publishes.
```xml
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
```
The answer, from [`../docs/output/01-default-exposure.txt`](output/01-default-exposure.txt):
```json
{"_links":{"self":{...},"health":{...},"health-path":{...}}}
```
One endpoint. `GET /actuator/env` returns **404**, not 403 — it was discovered, it exists as a
JMX endpoint, and it is simply not mapped onto HTTP.
That distinction matters more than it looks:
- **Discovery** — Actuator finds every `@Endpoint` bean on the classpath.
- **Access** (`management.endpoint.<id>.access`) — whether the endpoint may be operated at all.
Defaults to `unrestricted` for everything except `heapdump` and `shutdown`.
- **Exposure** (`management.endpoints.web.exposure.include`) — whether it is mapped onto HTTP.
Defaults to `health` only.
All three have to line up. A 404 from an Actuator path tells you nothing about whether the
endpoint is enabled, and people read it as "it's off" when it is often "it's on, over JMX".
## The default health body
```json
{"groups":["liveness","readiness"],"status":"DOWN"}
```
Two things to notice.
**The groups are there by default.** In Spring Boot 3 the liveness and readiness probes only
appeared when you asked for them or when Boot detected Kubernetes. In Boot 4 they are enabled
out of the box — see [chapter 02](02-boot-4-changes.md).
**`show-details` defaults to `never`**, so even an authenticated caller sees a bare status. That
is a sensible default and almost everyone overrides it to `always` without thinking about who
can reach the endpoint. [Chapter 04](04-securing-actuator.md) covers the middle option.
**The status is `DOWN`** because this repository registers a Kafka indicator and there is no
broker. One custom indicator that touches a third party is all it takes to turn the default
`/actuator/health` red — and that URL is what most Kubernetes manifests point their readiness
*and* liveness probes at. [Chapter 07](07-groups-and-probes.md) is about not doing that.
---
[← README](../README.md) · **01** · [02 What changed in Spring Boot 4 →](02-boot-4-changes.md)

107
docs/02-boot-4-changes.md Normal file
View File

@@ -0,0 +1,107 @@
[← 01 What Actuator exposes](01-what-actuator-exposes.md) · **02 · What changed in Spring Boot 4** · [03 The endpoint catalogue →](03-endpoint-catalogue.md)
# 02 — What changed in Spring Boot 4
Every claim below was checked against the 4.1.1 jars with `javap`, or against the 3.5.16 jars
for the "before" side. None of it came from a migration blog.
## `HealthIndicator` moved module and package
This is the one that breaks every custom health indicator ever written.
```
Boot 3.5.16 org.springframework.boot.actuate.health.HealthIndicator (spring-boot-actuator)
Boot 4.1.1 org.springframework.boot.health.contributor.HealthIndicator (spring-boot-health)
```
`org.springframework.boot.actuate.health` does not exist in Boot 4.1.1 at all — not deprecated,
absent. `Health`, `Status`, `AbstractHealthIndicator`, `HealthContributor` and the composites
all moved with it, into a new `spring-boot-health` module.
Two neighbouring packages in the same new module are worth knowing:
- `org.springframework.boot.health.application` — `DiskSpaceHealthIndicator`,
`LivenessStateHealthIndicator`, `ReadinessStateHealthIndicator`, `SslHealthIndicator`
- `org.springframework.boot.health.registry` — `HealthContributorRegistry`, which is how
[`DiagnosticsEndpoint`](../src/main/java/com/ankurm/actuator/web/DiagnosticsEndpoint.java)
enumerates contributors at runtime
## The interface method was renamed
```java
// Boot 3.5.16
public interface HealthIndicator extends HealthContributor {
default Health getHealth(boolean includeDetails);
Health health();
}
// Boot 4.1.1
public interface HealthIndicator extends HealthContributor {
default Health health(boolean includeDetails);
Health health();
}
```
If you overrode `getHealth(boolean)` — which people do to control detail rendering — it now
silently stops being an override. Add `@Override` and let the compiler find it.
`Health` also stopped extending `HealthComponent`; that class is gone from Boot 4.1.1 entirely.
## `EndpointRequest` moved too
```
Boot 3.5.16 org.springframework.boot.actuate.autoconfigure.security.servlet.EndpointRequest
Boot 4.1.1 org.springframework.boot.security.autoconfigure.actuate.web.servlet.EndpointRequest
```
Same class, same static methods (`toAnyEndpoint()`, `to(Class...)`, `to(String...)`,
`toLinks()`, `toAdditionalPaths(...)`), new package, and it now ships in `spring-boot-security`
rather than `spring-boot-actuator-autoconfigure`. The reactive variant made the equivalent move.
## `RestClient` is not in the web starter any more
Boot 4 split the framework into fine-grained modules, and `RestClient` auto-configuration went
with it. Adding `spring-boot-starter-webmvc` and injecting `RestClient.Builder` fails at
startup:
```
Parameter 0 of constructor in com.ankurm.actuator.health.ExternalApiHealthIndicator
required a bean of type 'org.springframework.web.client.RestClient$Builder' that could not be found.
```
The fix is one dependency:
```xml
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-restclient</artifactId>
</dependency>
```
This cost a startup cycle while building this repository, which is why it is documented here
rather than glossed over.
## Liveness and readiness probes are on by default
From the 4.0 migration guide, and confirmed by running the app with no configuration at all —
`/actuator/health` reports `"groups":["liveness","readiness"]` out of the box. Turn them off
with `management.endpoint.health.probes.enabled=false` if you genuinely do not want them.
## Nullability annotations
Actuator endpoint parameters can no longer use `org.springframework.lang.Nullable` to mark a
parameter optional. Migrate to `org.jspecify.annotations.Nullable`.
## There is no Micrometer 2
Several migration write-ups claim Boot 4 replaces "legacy Actuator endpoints" with a
"Micrometer 2 observability stack". Spring Boot 4.1.1 resolves **Micrometer 1.17.1**
(`docs/output/00-versions.txt`), and `maven-metadata.xml` for `micrometer-core` shows 1.17.1 as
the newest GA with 1.18.0-M1 as a milestone. Metrics *did* move to their own module
(`spring-boot-micrometer-metrics`, which `spring-boot-starter-actuator` still pulls in), and
there is a new `spring-boot-starter-opentelemetry`. Neither of those is a major Micrometer
release.
---
[← 01](01-what-actuator-exposes.md) · **02** · [03 The endpoint catalogue →](03-endpoint-catalogue.md)

89
docs/03-endpoint-catalogue.md Normal file
View File

@@ -0,0 +1,89 @@
[← 02 Boot 4 changes](02-boot-4-changes.md) · **03 · The endpoint catalogue** · [04 Securing Actuator →](04-securing-actuator.md)
# 03 — The endpoint catalogue
The table in the [README](../README.md#endpoints) is the reference. This chapter covers the
three endpoints people get wrong.
## `env` does not leak values
The oldest Actuator scare story is "`/actuator/env` dumps your database password". In Spring
Boot 4 it does not. From [`output/03-open-actuator-leak.txt`](output/03-open-actuator-leak.txt),
with no credentials sent at all:
```json
{"source":"Config resource 'class path resource [application.yaml]' ...","value":"******"}
```
That is `spring.datasource.password`, and it is also `acme.partner.credential` — a property
whose name matches none of the classic `password`/`secret`/`token`/`key` patterns. Both masked.
Masking is not pattern matching on key names. It is `management.endpoint.env.show-values`,
which defaults to `never`. Set it to `always` and both come back in plaintext:
```
acme.partner.credential S3CRET-partner-credential
spring.datasource.password not-a-real-password-but-watch-what-/actuator/env-does-with-it
spring.security.user.password ops-password
```
So the risk with `env` is real but it is a *configuration* risk, not a default. The `origin`
field is also worth knowing about — it reports the exact file and line a property came from,
which is genuinely the fastest way to answer "where is this value coming from" in a service
with six property sources.
## `heapdump` is not exposed by `include: "*"`
This surprises people who learned Actuator on Boot 2.
```
$ curl -s -o /dev/null -w '%{http_code}' http://localhost:8080/actuator/heapdump
404
```
…with `management.endpoints.web.exposure.include: "*"` and a `permitAll` security chain.
`management.endpoint.heapdump.access` defaults to `none`, and exposure does not override
access. Reading Spring Boot's own `spring-configuration-metadata.json` for every
`management.endpoint.*.access` key gives exactly two endpoints defaulting to `none`:
| Endpoint | `access` default |
|---|---|
| `heapdump` | `none` |
| `shutdown` | `none` |
| everything else | `unrestricted` |
Turn it on and the story changes completely —
[`output/04-heapdump-leak.txt`](output/04-heapdump-leak.txt):
```
status=200 bytes=59065395 type=application/octet-stream
$ strings /tmp/heap.hprof | grep -c 'S3CRET-partner-credential'
1
$ strings /tmp/heap.hprof | grep -o 'not-a-real-password[^"]*' | head -1
not-a-real-password-but-watch-what-/actuator/env-does-with-it
```
59 MB containing, in plaintext, both properties that `/actuator/env` had just masked.
Sanitisation is a rendering feature of one endpoint. It is not a security boundary.
## `loggers` is a write endpoint
`GET /actuator/loggers/{name}` is harmless. `POST` is not:
```
$ curl -X POST -H 'Content-Type: application/json' \
-d '{"configuredLevel":"TRACE"}' /actuator/loggers/org.springframework
status=204
```
204, no credentials. An attacker who can do that has a denial-of-service primitive (TRACE on a
busy service will fill a disk) and, depending on your logging configuration, a way to get
request bodies and headers written to a file they may be able to read through another channel.
`beans` (426 entries here) and `mappings` (30 servlet mappings) are read-only but are pure
reconnaissance — they describe your entire application's shape.
---
[← 02](02-boot-4-changes.md) · **03** · [04 Securing Actuator →](04-securing-actuator.md)

122
docs/04-securing-actuator.md Normal file
View File

@@ -0,0 +1,122 @@
[← 03 Endpoint catalogue](03-endpoint-catalogue.md) · **04 · Securing Actuator** · [05 Custom health indicators →](05-custom-health-indicators.md)
# 04 — Securing Actuator
Two mechanisms, and you want both. Network isolation decides who can reach the port;
authorisation decides who can use it. Neither is a substitute for the other.
## The separate management port
[`SecuredActuatorConfig`](../src/main/java/com/ankurm/actuator/config/SecuredActuatorConfig.java)
handles authorisation. `application-mgmtport.yaml` handles reachability:
```yaml
management:
server:
port: 9001
address: 127.0.0.1
endpoints:
web:
base-path: /manage
```
From [`output/06-management-port.txt`](output/06-management-port.txt):
```
GET :8080/actuator/health 401 <- application chain, no Actuator here
GET :8080/orders/count 200
GET :9001/manage/health 503
GET :9001/actuator/health 404 <- base-path moved
GET :9001/orders/count 404 <- separate context, no app controllers
```
```
LISTEN *:8080
LISTEN [::ffff:127.0.0.1]:9001
```
The last two lines are the argument. The management port runs a **separate application context**
with its own `DispatcherServlet`, so it cannot serve your controllers, and binding it to
loopback (or simply not listing it as a Kubernetes Service port) makes Actuator unreachable
from the internet by routing rather than by a rule somebody has to keep correct.
This is defence in depth, not a replacement for authorisation. Anything running in the same pod
or on the same host still reaches 9001.
## Match on `EndpointRequest`, not on a path
```java
http.securityMatcher(EndpointRequest.toAnyEndpoint())
.authorizeHttpRequests((requests) -> requests
.requestMatchers(EndpointRequest.to(HealthEndpoint.class, InfoEndpoint.class)).permitAll()
.anyRequest().hasRole("ACTUATOR"))
```
A rule written as `requestMatchers("/actuator/**")` stops matching the moment somebody sets
`management.endpoints.web.base-path` — as the `mgmtport` profile does. The endpoint moves, the
rule does not follow, and the endpoint is now unprotected with no error anywhere. `EndpointRequest`
asks the endpoint registry where things actually are.
## Order the chains
`@Order(1)` on the Actuator chain, `@Order(2)` on the application chain. Without explicit
ordering, whichever chain Spring registers first wins for a given request, and an application
chain ending in `permitAll()` will happily swallow `/actuator/**`.
## The resulting matrix
From [`output/05-secured-matrix.txt`](output/05-secured-matrix.txt), with exposure set to `"*"`:
| Request | Anonymous | `ops` / ROLE_ACTUATOR | Wrong password |
|---|---|---|---|
| `/actuator/health` | 503 | 503 | — |
| `/actuator/info` | 200 | — | — |
| `/actuator/env` | 401 | 200 | 401 |
| `/actuator/beans` | 401 | 200 | — |
| `/actuator/threaddump` | 401 | 200 | — |
| `/actuator` (links) | 401 | — | — |
| `/orders/count` | 200 | — | — |
Exposure is `"*"` and nothing leaks. That is the point: **exposure is not access control.**
Use exposure to decide what exists, and the security chain to decide who may use it.
## `show-details: when-authorized`
The third option, and the one to reach for. Anonymous:
```json
{"groups":["liveness","readiness"],"status":"DOWN"}
```
Authenticated with `ROLE_ACTUATOR`: the full nine-component breakdown, naming `kafka` as the
failing one. Same URL, same status code, different body. Your load balancer gets the signal it
needs; an anonymous prober learns that something is wrong but not which of your dependencies to
attack next.
```yaml
management:
endpoint:
health:
show-details: when-authorized
roles: ACTUATOR
```
## Make the Actuator chain stateless and CSRF-free
Spring Boot's auto-configured security is browser-shaped: it enables CSRF protection. Actuator
is a machine-to-machine API, and `POST /actuator/loggers/{name}` from a script will be rejected
before it reaches the endpoint.
While building this repository that showed up as a `401` on `POST /stub/upstream/mode` while
`GET /orders/count` with the *same credentials* returned `200` — a genuinely confusing pair of
results that has nothing to do with the credentials. See
[`DemoSecurityConfig`](../src/main/java/com/ankurm/actuator/config/DemoSecurityConfig.java).
```java
.csrf((csrf) -> csrf.disable())
.sessionManagement((s) -> s.sessionCreationPolicy(SessionCreationPolicy.STATELESS))
```
---
[← 03](03-endpoint-catalogue.md) · **04** · [05 Custom health indicators →](05-custom-health-indicators.md)

112
docs/05-custom-health-indicators.md Normal file
View File

@@ -0,0 +1,112 @@
[← 04 Securing Actuator](04-securing-actuator.md) · **05 · Custom health indicators** · [06 Failure modes →](06-health-indicator-failure-modes.md)
# 05 — Custom health indicators
Three indicators, three different jobs.
| Bean | Class | Checks |
|---|---|---|
| `ordersDatabase` | [`OrdersDatabaseHealthIndicator`](../src/main/java/com/ankurm/actuator/health/OrdersDatabaseHealthIndicator.java) | the query the application depends on |
| `kafka` | [`KafkaHealthIndicator`](../src/main/java/com/ankurm/actuator/health/KafkaHealthIndicator.java) | `AdminClient.describeCluster` |
| `externalApi` | [`ExternalApiHealthIndicator`](../src/main/java/com/ankurm/actuator/health/ExternalApiHealthIndicator.java) | an HTTP dependency, with a budget |
## The shape
```java
import org.springframework.boot.health.contributor.AbstractHealthIndicator;
import org.springframework.boot.health.contributor.Health;
@Component("ordersDatabase")
public class OrdersDatabaseHealthIndicator extends AbstractHealthIndicator {
@Override
protected void doHealthCheck(Health.Builder builder) throws Exception {
// throw, or call builder.up() / down() / status("DEGRADED")
}
}
```
Note the package — `org.springframework.boot.health.contributor`, new in Boot 4. See
[chapter 02](02-boot-4-changes.md).
`AbstractHealthIndicator` over the bare interface, because it catches your exceptions and turns
them into `DOWN` with the error recorded, rather than letting one throwing indicator take down
the whole `/actuator/health` response.
## The bean name is the JSON key
`@Component("ordersDatabase")` produces `"ordersDatabase"` in the response. Boot strips a
trailing `HealthIndicator` from the bean name, so `ordersDatabaseHealthIndicator` gives the same
key. Rename the bean and every dashboard, alert and health-group `include:` that referenced the
old key silently stops matching — which is why
[`HealthGroupTests`](../src/test/java/com/ankurm/actuator/HealthGroupTests.java) asserts on the
names.
## Check what the application needs, not what the connection can do
Boot's built-in `db` indicator runs a validation query and answers "can I reach the database".
That is rarely the question. `ordersDatabase` runs the query the service actually depends on and
adds a latency judgement:
```json
{"details":{"orders":3,"queryMs":0,"slowThresholdMs":250},"status":"UP"}
```
It returns a custom `DEGRADED` status above the threshold. Custom statuses are legal;
`management.endpoint.health.status.order` and `.http-mapping` control how they aggregate and
what HTTP code they produce. An unmapped custom status aggregates as `UNKNOWN` and maps to 200,
which is usually not what you meant.
## Every indicator needs a budget
All three carry an explicit timeout, and the reason is [chapter 06](06-health-indicator-failure-modes.md).
- `ordersDatabase` — `jdbc.setQueryTimeout(2)`
- `externalApi` — connect and read timeouts of 750 ms on the `RestClient` request factory
- `kafka` — four separate Kafka bounds plus an outer `KafkaFuture.get(timeout)`
## Credentials
`externalApi` sends HTTP Basic. The first captured run of this repository did not, and reported:
```json
{"error":"HttpClientErrorException$Unauthorized: 401 : [no body]","status":"DOWN"}
```
which looks exactly like an upstream outage and was a missing header. If your indicator
authenticates, make the failure message distinguish "they are down" from "we are unauthorised" —
future you will be reading it at 3am.
## Reuse the client
`KafkaHealthIndicator` creates one `AdminClient` in its constructor and closes it in
`close()`. Creating one per probe opens a fresh set of broker connections on every probe
interval; across a fleet of any size that is a denial-of-service against your own brokers.
## What it all looks like
[`output/07-custom-health-indicators.txt`](output/07-custom-health-indicators.txt) has the full
nine-component response. Flipping the stub upstream to `DOWN` and changing nothing else:
```json
{
"status": "DOWN",
"externalApi": {
"details": {
"error": "HttpServerErrorException$ServiceUnavailable: 503 : \"upstream unavailable\"",
"url": "http://localhost:8080/stub/upstream/ping",
"latencyMs": 101,
"timeoutMs": 750
},
"status": "DOWN"
}
}
```
`/actuator/health` now answers 503. If that URL is your readiness probe, every pod in the
deployment just left the load balancer because a third party had a bad minute.
[Chapter 07](07-groups-and-probes.md) fixes that.
---
[← 04](04-securing-actuator.md) · **05** · [06 Failure modes →](06-health-indicator-failure-modes.md)

100
docs/06-health-indicator-failure-modes.md Normal file
View File

@@ -0,0 +1,100 @@
[← 05 Custom health indicators](05-custom-health-indicators.md) · **06 · Health indicator failure modes** · [07 Groups and probes →](07-groups-and-probes.md)
# 06 — Health indicator failure modes
## The 60-second Kafka health check
This is the most useful number in the repository. The textbook Kafka health check:
```java
AdminClient admin = AdminClient.create(Map.of(BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"));
admin.describeCluster().nodes().get();
```
Against an unreachable broker, from
[`output/09-kafka-timeout.txt.naive`](output/09-kafka-timeout.txt.naive):
```json
{"details":{"tuned":false,"error":"TimeoutException: Timed out waiting for a node assignment. Call: listNodes","probeMs":60002,"budgetMs":180000},"status":"DOWN"}
wall clock: 60.2 s
```
**60.002 seconds.** The outer budget was 180 s, so it was not the binding constraint — Kafka's
own `default.api.timeout.ms` was. These are the `AdminClientConfig` defaults, read from
`AdminClientConfig.configDef()` on kafka-clients 4.2.1:
| Property | Default |
|---|---|
| `request.timeout.ms` | 30000 |
| `default.api.timeout.ms` | **60000** |
| `socket.connection.setup.timeout.ms` | 10000 |
| `retries` | 2147483647 |
| `metadata.recovery.strategy` | `rebootstrap` |
The trap is that `request.timeout.ms` is the one everybody sets. It bounds a single attempt, and
with `retries` at `Integer.MAX_VALUE` the client simply attempts again until
`default.api.timeout.ms` fires. Set only `request.timeout.ms` and your health check still blocks
for a minute.
The tuned version, same broker, same absence of it —
[`output/09-kafka-timeout.txt.tuned`](output/09-kafka-timeout.txt.tuned):
```json
{"details":{"tuned":true,"error":"TimeoutException: null","probeMs":1500,"budgetMs":1500},"status":"DOWN"}
wall clock: 1.6 s
```
Forty times faster, from setting four properties and one `KafkaFuture.get(timeout)`.
Why this matters beyond the number: your probe interval is probably 10 seconds. A check that
takes 60 seconds means six probes are in flight at once, each holding a container worker thread,
for as long as the broker is down. That is how a Kafka outage becomes an application outage.
## The rebootstrap storm
Kafka 4 defaults `metadata.recovery.strategy` to `rebootstrap`. A long-lived `AdminClient`
pointed at a dead broker keeps a background thread retrying, and it logs. The first run of this
repository produced **192** `Rebootstrapping with Cluster(id = null, ...)` lines in a few
seconds, and the sandbox ran out of memory shortly after.
`metadata.recovery.strategy: none` plus a sane `reconnect.backoff.max.ms` stops it. Whatever you
choose, know that the AdminClient's background thread is doing work whether or not anyone is
calling your health endpoint.
## The unbounded HTTP call
`ExternalApiHealthIndicator` sets connect and read timeouts of 750 ms. Against a stub that
sleeps 30 seconds — [`output/10-slow-upstream.txt`](output/10-slow-upstream.txt):
```json
{"details":{"error":"ResourceAccessException: I/O error on GET request ...: Read timed out","latencyMs":753,"timeoutMs":750},"status":"DOWN"}
wall clock: 0.84 s
```
`latencyMs: 753` against `timeoutMs: 750`. Remove `setReadTimeout` and that becomes 30 seconds,
holding a Tomcat worker the entire time.
The default for `SimpleClientHttpRequestFactory` — and for `HttpURLConnection` underneath it —
is no timeout at all. Not a long timeout. None.
## Rules of thumb
- **Every remote call in a health indicator needs an explicit timeout**, and the total of all of
them should be comfortably under your probe interval.
- **Set the API-level timeout, not just the request-level one.** Kafka is the sharpest example
but the pattern recurs — clients with internal retry loops need an outer bound.
- **Fail fast and fail loudly.** A `DOWN` with a useful `error` detail beats a probe that hangs.
- **Health checks should be cheap.** They run on every instance on every probe interval, forever.
- **Do not check dependencies you cannot act on.** If a third party being down does not change
what this instance should do, do not put it in a health indicator at all.
There is a cache if you need it: `management.endpoint.health.cache.time-to-live`. Reach for it
only after you have fixed the timeouts — caching a 60-second check gives you a fast endpoint
serving a stale answer, which is worse than a slow honest one.
---
[← 05](05-custom-health-indicators.md) · **06** · [07 Groups and probes →](07-groups-and-probes.md)

130
docs/07-groups-and-probes.md Normal file
View File

@@ -0,0 +1,130 @@
[← 06 Failure modes](06-health-indicator-failure-modes.md) · **07 · Groups, probes and Kubernetes** · [08 The diagnostics endpoint →](08-diagnostics.md)
# 07 — Groups, probes and Kubernetes
## The failure this prevents
A third-party API goes down. Your `externalApi` indicator reports `DOWN`. `/actuator/health`
aggregates to `DOWN` and answers 503. Your Kubernetes manifest points **both** probes at it:
```yaml
livenessProbe: { httpGet: { path: /actuator/health, port: 8080 } }
readinessProbe: { httpGet: { path: /actuator/health, port: 8080 } }
```
The readiness failure is correct — this instance cannot serve. The liveness failure is a
catastrophe: the kubelet kills the container, every replica fails the same probe at the same
time, and the whole deployment enters a restart loop. A partial outage in someone else's system
has become a total outage in yours, and the restarts make recovery slower because every fresh
JVM has to warm up while the upstream is still failing.
**Liveness answers "is this process broken beyond recovery?"** Almost nothing external belongs
in it, because restarting cannot fix an external problem.
**Readiness answers "should this instance receive traffic right now?"** Dependencies belong here.
## The configuration
From `application-groups.yaml`:
```yaml
management:
endpoint:
health:
group:
liveness:
include: livenessState,diskSpace
readiness:
include: readinessState,ordersDatabase,externalApi
startup:
include: ordersDatabase
probes:
enabled: true
```
Each group gets its own path: `/actuator/health/liveness`, `/actuator/health/readiness`,
`/actuator/health/startup`. Groups can carry their own `show-details` too.
## The evidence
[`output/08-groups-and-probes.txt`](output/08-groups-and-probes.txt):
```
--- baseline: upstream UP ---
GET /actuator/health HTTP 503
GET /actuator/health/liveness HTTP 200
GET /actuator/health/readiness HTTP 200
GET /actuator/health/startup HTTP 200
--- upstream DOWN ---
GET /actuator/health HTTP 503
GET /actuator/health/liveness HTTP 200
GET /actuator/health/readiness HTTP 503
GET /actuator/health/startup HTTP 200
```
Liveness stayed 200 across the outage. Readiness moved. Kubernetes removes this instance from
the Service endpoints and leaves the process alone; when the upstream recovers, readiness goes
green and traffic returns without a single restart.
Note the aggregate `/actuator/health` was 503 in *both* columns — the Kafka indicator is down
throughout, since this repository runs no broker. That is exactly why you should not point a
probe at the aggregate: it is the union of everything, and it tells the orchestrator nothing
actionable.
## The manifest
```yaml
livenessProbe:
httpGet: { path: /actuator/health/liveness, port: 9001 }
periodSeconds: 10
failureThreshold: 3
readinessProbe:
httpGet: { path: /actuator/health/readiness, port: 9001 }
periodSeconds: 5
failureThreshold: 2
startupProbe:
httpGet: { path: /actuator/health/startup, port: 9001 }
periodSeconds: 5
failureThreshold: 30
```
Port 9001 is the management port from [chapter 04](04-securing-actuator.md). The kubelet reaches
it inside the pod network; the ingress does not.
The `startupProbe` matters more than people think. Without one, a JVM that takes 45 seconds to
warm up under `failureThreshold: 3` and `periodSeconds: 10` gets killed at 30 seconds, forever,
and the symptom is a crash-loop with no error in the logs.
## `readinessState`, and shutting down cleanly
`readinessState` is Boot's own indicator, driven by `ApplicationAvailability`. During graceful
shutdown Boot flips it to `REFUSING_TRAFFIC` before the server stops accepting, so readiness
goes red while in-flight requests finish. Pair it with:
```yaml
server:
shutdown: graceful
spring:
lifecycle:
timeout-per-shutdown-phase: 30s
```
You can also drive it yourself by publishing an `AvailabilityChangeEvent` — useful for taking an
instance out of rotation before a risky migration.
## Which dependencies belong in readiness
Ask: *if this is down, can this instance still serve any useful request?*
- Primary database → yes, readiness.
- Kafka, for a service whose only job is consuming → yes, readiness.
- Kafka, for a service that also serves reads → probably not; degrade rather than withdraw.
- A recommendations API you fall back to a static list for → no. Do not check it at all.
Every dependency in readiness is a dependency that can take your service out of rotation. That
list should be shorter than your instinct suggests.
---
[← 06](06-health-indicator-failure-modes.md) · **07** · [08 The diagnostics endpoint →](08-diagnostics.md)

95
docs/08-diagnostics.md Normal file
View File

@@ -0,0 +1,95 @@
[← 07 Groups and probes](07-groups-and-probes.md) · **08 · The diagnostics endpoint** · [09 Testing Actuator →](09-testing-actuator.md)
# 08 — The diagnostics endpoint
[`DiagnosticsEndpoint`](../src/main/java/com/ankurm/actuator/web/DiagnosticsEndpoint.java) is a
custom `@Endpoint(id = "diag")` that prints the Actuator state no configuration file will tell
you: which endpoints were actually discovered and mapped onto HTTP, what paths and methods each
publishes, and which health contributors are registered.
**Delete it before you ship.** The list of exposed endpoints is itself reconnaissance.
## Why it exists
Reading `application.yaml` and predicting exposure is how people ship `/actuator/heapdump` to
the internet. `management.endpoints.web.exposure.include: "*"` looks like it exposes everything;
it does not expose `heapdump` or `shutdown` ([chapter 03](03-endpoint-catalogue.md)). Conditional
endpoints like `httpexchanges` and `startup` are absent unless a specific bean exists. Ask the
running application instead of predicting.
## Writing a custom endpoint
```java
@Component
@Endpoint(id = "diag")
public class DiagnosticsEndpoint {
@ReadOperation
public Map<String, Object> diagnostics() { ... }
}
```
`@ReadOperation` maps to GET, `@WriteOperation` to POST, `@DeleteOperation` to DELETE. The id
must be lowercase alphanumeric; it becomes the path segment and the JMX name. You still have to
expose it — `include: health,info,diag` or `"*"`.
The interesting part is the three beans it injects:
| Bean | Gives you |
|---|---|
| `WebEndpointsSupplier` | every `ExposableWebEndpoint`, with its `WebOperation` predicates |
| `HealthContributorRegistry` | the live contributor tree, composites included |
| `Environment` | the effective values of the `management.*` keys |
`HealthContributorRegistry` is in `org.springframework.boot.health.registry` — new in Boot 4
([chapter 02](02-boot-4-changes.md)). It iterates as `HealthContributors.Entry`, and nested
`HealthContributors` are composites, which is why the collection is recursive.
## The output
From [`output/02-endpoint-catalogue.txt`](output/02-endpoint-catalogue.txt):
```json
{
"activeProfiles": ["exposeall", "open"],
"serverPort": "8080",
"managementPort": "(same as server.port)",
"managementBasePath": "/actuator",
"exposureInclude": "*",
"healthShowDetails": "always",
"exposedWebEndpointCount": 14,
"exposedWebEndpoints": {
"loggers": ["GET loggers", "GET loggers/{name}", "POST loggers/{name}"],
"threaddump": ["GET threaddump", "GET threaddump"],
...
},
"healthContributors": [
"db (DataSourceHealthIndicator)",
"diskSpace (DiskSpaceHealthIndicator)",
"externalApi (ExternalApiHealthIndicator)",
"kafka (KafkaHealthIndicator)",
"livenessState (LivenessStateHealthIndicator)",
"ordersDatabase (OrdersDatabaseHealthIndicator)",
"ping (PingHealthIndicator)",
"readinessState (ReadinessStateHealthIndicator)",
"ssl (SslHealthIndicator)"
]
}
```
Two things worth pausing on.
`"loggers"` shows the `POST` operation explicitly. That is the write endpoint from
[chapter 03](03-endpoint-catalogue.md), visible in a list rather than remembered.
`"threaddump"` appears **twice** with the same path. It publishes two operations that differ
only in what they produce — `application/json` and `text/plain`. Content negotiation, not a bug.
Run the same endpoint under the `mgmtport` profile and `managementPort` reads `9001`,
`managementBasePath` reads `/manage`, and the endpoint count is unchanged — the endpoints moved,
they did not disappear. That is the fact that breaks path-string security rules
([chapter 04](04-securing-actuator.md)).
---
[← 07](07-groups-and-probes.md) · **08** · [09 Testing Actuator →](09-testing-actuator.md)

95
docs/09-testing-actuator.md Normal file
View File

@@ -0,0 +1,95 @@
[← 08 The diagnostics endpoint](08-diagnostics.md) · **09 · Testing Actuator** · [README →](../README.md)
# 09 — Testing Actuator
Eight tests, and none of them assert the happy path. They pin the behaviour that would be
expensive to rediscover after an upgrade.
## Pin the surprises
[`ActuatorExposureTests`](../src/test/java/com/ankurm/actuator/ActuatorExposureTests.java) runs
with `exposeall,secured` — the widest exposure with real security — and asserts:
```java
@Test
void heapdumpIsNotExposedEvenWithWildcardExposure() throws Exception {
mvc().perform(get("/actuator/heapdump")).andExpect(status().isNotFound());
}
```
If a future Boot release changes the `access` default for `heapdump`, or if somebody adds
`management.endpoint.heapdump.access=unrestricted` to a shared config, this test is where you
find out. Same for `shutdown`.
```java
@Test
void envMasksEveryValueRegardlessOfKeyName() throws Exception {
// acme.partner.credential matches no password/secret/token pattern.
... .andExpect(jsonPath("$.property.value").value("******"));
}
@Test
void anonymousHealthCarriesNoComponentBreakdown() throws Exception {
mvc().perform(get("/actuator/health"))
.andExpect(jsonPath("$.components").doesNotExist());
}
```
That last one is the `show-details: when-authorized` contract from
[chapter 04](04-securing-actuator.md), asserted rather than assumed. It is a one-line
configuration change away from leaking your dependency map to anonymous callers.
## Test the operational contract, not the indicator
[`HealthGroupTests`](../src/test/java/com/ankurm/actuator/HealthGroupTests.java):
```java
@Test
void upstreamOutageMovesTheExternalApiIndicatorButNotLiveness() {
upstream.set(UpstreamState.Mode.DOWN);
assertThat(statusOf("externalApi")).isEqualTo(Status.DOWN);
assertThat(statusOf("livenessState")).isEqualTo(Status.UP);
}
```
That is [chapter 07](07-groups-and-probes.md) as an executable assertion. It fails if someone
adds `externalApi` to the liveness group, which is the exact change that causes a fleet-wide
restart loop.
```java
@Test
void theExternalApiIndicatorRespectsItsTimeoutBudget() {
upstream.set(UpstreamState.Mode.SLOW); // the stub sleeps 30s
...
assertThat(elapsedMs).isLessThan(5_000L);
}
```
Fails if someone removes the read timeout. A generous bound, because timing assertions on CI are
how you get flaky suites — but 5 s versus 30 s is a wide enough gap to be safe.
## `DEFINED_PORT`, and why
```java
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
```
Not the default `MOCK` environment. `ExternalApiHealthIndicator` makes a real HTTP call to this
application's own stub controller, so a servlet container has to be listening on the port it was
configured with. Under `MOCK` the indicator reports `DOWN` with a connection error and every
assertion about upstream state is meaningless — which is precisely what happened on the first
run of this suite.
`RANDOM_PORT` does not help here: the indicator resolves its URL at bean construction, before
the server binds, so `local.server.port` is not available to it.
## What is not tested
There is no test for the 60-second Kafka timeout. A test that takes a minute to pass does not
belong in a suite people run before pushing —
[`scripts/demo-kafka-timeout.sh`](../scripts/demo-kafka-timeout.sh) captures it instead, and the
transcript is committed. Some evidence is better as a script than as an assertion.
---
[← 08](08-diagnostics.md) · **09** · [README →](../README.md)

23
docs/output/00-versions.txt Normal file
View File

@@ -0,0 +1,23 @@
$ java -version
openjdk version "25.0.4.1" 2026-08-18 LTS
OpenJDK Runtime Environment Temurin-25.0.4.1+1 (build 25.0.4.1+1-LTS)
OpenJDK 64-Bit Server VM Temurin-25.0.4.1+1 (build 25.0.4.1+1-LTS, mixed mode, sharing)
$ mvn -v | head -3
Apache Maven 3.9.11 (3e54c93a704957b63ee3494413a2b544fd3d825b)
Maven home: /tmp/tools/apache-maven-3.9.11
Java version: 25.0.4.1, vendor: Eclipse Adoptium, runtime: /tmp/tools/jdk-25.0.4.1+1
$ mvn dependency:list -- the resolved versions behind spring-boot-starter-parent 4.1.1
com.h2database:h2 2.4.240
io.micrometer:micrometer-core 1.17.1
io.micrometer:micrometer-registry-prometheus 1.17.1
org.apache.kafka:kafka-clients 4.2.1
org.springframework.boot:spring-boot 4.1.1
org.springframework.boot:spring-boot-actuator 4.1.1
org.springframework.boot:spring-boot-health 4.1.1
org.springframework.boot:spring-boot-restclient 4.1.1
org.springframework.security:spring-security-core 7.1.1
org.springframework.security:spring-security-web 7.1.1
org.springframework:spring-core 7.0.9
org.springframework:spring-web 7.0.9

35
docs/output/01-default-exposure.txt Normal file
View File

@@ -0,0 +1,35 @@
### Spring Boot Actuator, starter added, ZERO management.* configuration
$ curl -s -u ops:ops-password http://localhost:8080/actuator
{
"_links": {
"self": {
"href": "http://localhost:8080/actuator",
"templated": false
},
"health": {
"href": "http://localhost:8080/actuator/health",
"templated": false
},
"health-path": {
"href": "http://localhost:8080/actuator/health/{*path}",
"templated": true
}
}
}
$ curl -s -o /dev/null -w '%{http_code}' -u ops:ops-password http://localhost:8080/actuator/env
404
404 = discovered but NOT exposed over HTTP. Exposure and existence are different things.
$ curl -s -u ops:ops-password http://localhost:8080/actuator/health
{
"groups": [
"liveness",
"readiness"
],
"status": "DOWN"
}
Only 'health' is web-exposed by default. show-details defaults to 'never', so even an
authenticated caller sees a bare status until you say otherwise.

80
docs/output/02-endpoint-catalogue.txt Normal file
View File

@@ -0,0 +1,80 @@
### Every web-exposed endpoint, read from the running application
### profiles: exposeall,open management.endpoints.web.exposure.include: "*"
$ curl -s http://localhost:8080/actuator/diag
{
"activeProfiles": [
"exposeall",
"open"
],
"serverPort": "8080",
"managementPort": "(same as server.port)",
"managementBasePath": "/actuator",
"exposureInclude": "*",
"exposureExclude": "(none)",
"healthShowDetails": "always",
"exposedWebEndpointCount": 14,
"exposedWebEndpoints": {
"beans": [
"GET beans"
],
"conditions": [
"GET conditions"
],
"configprops": [
"GET configprops",
"GET configprops/{prefix}"
],
"diag": [
"GET diag"
],
"env": [
"GET env",
"GET env/{toMatch}"
],
"health": [
"GET health",
"GET health/{*path}"
],
"info": [
"GET info"
],
"loggers": [
"GET loggers",
"GET loggers/{name}",
"POST loggers/{name}"
],
"mappings": [
"GET mappings"
],
"metrics": [
"GET metrics",
"GET metrics/{requiredMetricName}"
],
"prometheus": [
"GET prometheus"
],
"sbom": [
"GET sbom",
"GET sbom/{id}"
],
"scheduledtasks": [
"GET scheduledtasks"
],
"threaddump": [
"GET threaddump",
"GET threaddump"
]
},
"healthContributors": [
"db (DataSourceHealthIndicator)",
"diskSpace (DiskSpaceHealthIndicator)",
"externalApi (ExternalApiHealthIndicator)",
"kafka (KafkaHealthIndicator)",
"livenessState (LivenessStateHealthIndicator)",
"ordersDatabase (OrdersDatabaseHealthIndicator)",
"ping (PingHealthIndicator)",
"readinessState (ReadinessStateHealthIndicator)",
"ssl (SslHealthIndicator)"
]
}

39
docs/output/03-open-actuator-leak.txt Normal file
View File

@@ -0,0 +1,39 @@
### profiles: exposeall,open -- NO credentials are sent on any request below
--- 1. /actuator/env does NOT leak values in Spring Boot 4 ---
$ curl -s http://localhost:8080/actuator/env/spring.datasource.password | jq .property
{
"source": "Config resource 'class path resource [application.yaml]' via location 'optional:classpath:/'",
"value": "******"
}
$ curl -s http://localhost:8080/actuator/env/acme.partner.credential | jq .property
{
"source": "Config resource 'class path resource [application.yaml]' via location 'optional:classpath:/'",
"value": "******"
}
Note the second one. 'acme.partner.credential' matches none of the classic
password/secret/token key patterns, and it is still masked. Masking is driven by
management.endpoint.env.show-values, which defaults to 'never' - not by key names.
--- 2. /actuator/heapdump is NOT exposed by 'include: "*"' ---
$ curl -s -o /dev/null -w '%{http_code}' http://localhost:8080/actuator/heapdump
404
management.endpoint.heapdump.access defaults to 'none'. So does shutdown.
They are the only two endpoints that do.
--- 3. /actuator/loggers: an unauthenticated WRITE ---
$ curl -s -X POST -d '{"configuredLevel":"TRACE"}' -H 'Content-Type: application/json' http://localhost:8080/actuator/loggers/org.springframework
status=204
$ curl -s http://localhost:8080/actuator/loggers/org.springframework
{
"configuredLevel": "TRACE",
"effectiveLevel": "TRACE"
}
(level reset). An attacker who can flip your root logger to TRACE has both a
denial-of-service primitive and a way to get request bodies written to disk.
--- 4. /actuator/beans and /actuator/mappings: your whole application, described ---
$ curl -s http://localhost:8080/actuator/mappings | python3 -c 'count the URL patterns'
30 servlet mappings disclosed
426 beans disclosed, each with its type and dependencies

13
docs/output/04-heapdump-leak.txt Normal file
View File

@@ -0,0 +1,13 @@
### profiles: exposeall,open PLUS --management.endpoint.heapdump.access=unrestricted
$ curl -s -o /tmp/heap.hprof -w 'status=%{http_code} bytes=%{size_download} type=%{content_type}' http://localhost:8080/actuator/heapdump
status=200 bytes=59186852 type=application/octet-stream
$ strings /tmp/heap.hprof | grep -c 'S3CRET-partner-credential'
1
$ strings /tmp/heap.hprof | grep -o 'not-a-real-password[^"]*' | head -1
not-a-real-password-but-watch-what-/actuator/env-does-with-it!
/actuator/env masked both of these to ******.
/actuator/heapdump handed over the process memory that contains them in plaintext.
Sanitisation is a property-rendering feature. It is not a security boundary.

106
docs/output/05-secured-matrix.txt Normal file
View File

@@ -0,0 +1,106 @@
### profile: secured (SecuredActuatorConfig + application-secured.yaml)
### exposure is "*" - the security chain, not the exposure list, is what protects it
ANONYMOUS
GET /actuator/health 503
GET /actuator/info 200
GET /actuator/env 401
GET /actuator/beans 401
GET /actuator/threaddump 401
GET /actuator (the links index) 401
AUTHENTICATED as ops (ROLE_ACTUATOR)
GET /actuator/health 503
GET /actuator/env 200
GET /actuator/beans 200
GET /actuator/threaddump 200
WRONG PASSWORD
GET /actuator/env 401
--- health body, anonymous (show-details: when-authorized) ---
{
"groups": [
"liveness",
"readiness"
],
"status": "DOWN"
}
--- health body, authenticated as ROLE_ACTUATOR ---
{
"components": {
"db": {
"details": {
"database": "H2",
"validationQuery": "isValid()"
},
"status": "UP"
},
"diskSpace": {
"details": {
"total": 10213466112,
"free": 3877920768,
"threshold": 10485760,
"path": "/tmp/work/spring-boot-demo/.",
"exists": true
},
"status": "UP"
},
"externalApi": {
"details": {
"url": "http://localhost:8080/stub/upstream/ping",
"response": "pong",
"latencyMs": 4,
"timeoutMs": 750
},
"status": "UP"
},
"kafka": {
"details": {
"bootstrap": "localhost:9092",
"tuned": true,
"error": "TimeoutException: Timed out waiting for a node assignment. Call: listNodes",
"probeMs": 1500,
"budgetMs": 1500
},
"status": "DOWN"
},
"livenessState": {
"status": "UP"
},
"ordersDatabase": {
"details": {
"orders": 3,
"queryMs": 0,
"slowThresholdMs": 250
},
"status": "UP"
},
"ping": {
"status": "UP"
},
"readinessState": {
"status": "UP"
},
"ssl": {
"details": {
"expiringChains": [],
"invalidChains": [],
"validChains": []
},
"status": "UP"
}
},
"groups": [
"liveness",
"readiness"
],
"status": "DOWN"
}
Same endpoint, same status code, different body. An anonymous prober learns that the
service is unhealthy but not WHICH dependency is unhealthy.
--- the business endpoint is untouched by the actuator chain ---
GET /orders/count (anonymous) 200

101
docs/output/06-management-port.txt Normal file
View File

@@ -0,0 +1,101 @@
### profile: mgmtport
### management.server.port: 9001 / management.server.address: 127.0.0.1 / base-path: /manage
--- the application port no longer serves Actuator at all ---
GET :8080/actuator/health 401
GET :8080/manage/health 401
GET :8080/orders/count 200
--- the management port serves it on the new base path ---
GET :9001/manage/health 503
GET :9001/actuator/health 404
GET :9001/orders/count 404
Note the last line. The management context has its own DispatcherServlet and does NOT
see application controllers. That is the isolation you are paying for.
--- what the management context reports about itself ---
$ curl -s -u ops:ops-password http://127.0.0.1:9001/manage/diag
{
"activeProfiles": [
"mgmtport"
],
"serverPort": "8080",
"managementPort": "9001",
"managementBasePath": "/manage",
"exposureInclude": "*",
"exposureExclude": "(none)",
"healthShowDetails": "never",
"exposedWebEndpointCount": 14,
"exposedWebEndpoints": {
"beans": [
"GET beans"
],
"conditions": [
"GET conditions"
],
"configprops": [
"GET configprops",
"GET configprops/{prefix}"
],
"diag": [
"GET diag"
],
"env": [
"GET env",
"GET env/{toMatch}"
],
"health": [
"GET health",
"GET health/{*path}"
],
"info": [
"GET info"
],
"loggers": [
"GET loggers",
"GET loggers/{name}",
"POST loggers/{name}"
],
"mappings": [
"GET mappings"
],
"metrics": [
"GET metrics",
"GET metrics/{requiredMetricName}"
],
"prometheus": [
"GET prometheus"
],
"sbom": [
"GET sbom",
"GET sbom/{id}"
],
"scheduledtasks": [
"GET scheduledtasks"
],
"threaddump": [
"GET threaddump",
"GET threaddump"
]
},
"healthContributors": [
"db (DataSourceHealthIndicator)",
"diskSpace (DiskSpaceHealthIndicator)",
"externalApi (ExternalApiHealthIndicator)",
"kafka (KafkaHealthIndicator)",
"livenessState (LivenessStateHealthIndicator)",
"ordersDatabase (OrdersDatabaseHealthIndicator)",
"ping (PingHealthIndicator)",
"readinessState (ReadinessStateHealthIndicator)",
"ssl (SslHealthIndicator)"
]
}
--- listening sockets ---
$ ss -ltn | grep -E ':(8080|9001)'
LISTEN 0 100 *:8080 *:*
LISTEN 0 100 [::ffff:127.0.0.1]:9001 *:*
9001 is bound to 127.0.0.1 only. 8080 is bound to *. An ingress that forwards to 8080
cannot reach Actuator no matter how the security rules are written.

121
docs/output/07-custom-health-indicators.txt Normal file
View File

@@ -0,0 +1,121 @@
### profile: details (show-details: always, show-components: always)
--- upstream UP ---
$ curl -s -o /dev/null -w '%{http_code}' http://localhost:8080/actuator/health
HTTP 503
{
"components": {
"db": {
"details": {
"database": "H2",
"validationQuery": "isValid()"
},
"status": "UP"
},
"diskSpace": {
"details": {
"total": 10213466112,
"free": 3877675008,
"threshold": 10485760,
"path": "/tmp/work/spring-boot-demo/.",
"exists": true
},
"status": "UP"
},
"externalApi": {
"details": {
"url": "http://localhost:8080/stub/upstream/ping",
"response": "pong",
"latencyMs": 66,
"timeoutMs": 750
},
"status": "UP"
},
"kafka": {
"details": {
"bootstrap": "localhost:9092",
"tuned": true,
"error": "TimeoutException: null",
"probeMs": 1500,
"budgetMs": 1500
},
"status": "DOWN"
},
"livenessState": {
"status": "UP"
},
"ordersDatabase": {
"details": {
"orders": 3,
"queryMs": 0,
"slowThresholdMs": 250
},
"status": "UP"
},
"ping": {
"status": "UP"
},
"readinessState": {
"status": "UP"
},
"ssl": {
"details": {
"expiringChains": [],
"invalidChains": [],
"validChains": []
},
"status": "UP"
}
},
"groups": [
"liveness",
"readiness"
],
"status": "DOWN"
}
--- flip the upstream to DOWN, change nothing else ---
$ curl -s -X POST 'http://localhost:8080/stub/upstream/mode?value=down'
upstream mode = DOWN
HTTP 503
{
"status": "DOWN",
"externalApi": {
"details": {
"error": "org.springframework.web.client.HttpServerErrorException$ServiceUnavailable: 503 : \"upstream unavailable\"",
"url": "http://localhost:8080/stub/upstream/ping",
"latencyMs": 67,
"timeoutMs": 750
},
"status": "DOWN"
}
}
The aggregate went DOWN and /actuator/health now answers 503. If that URL is your
Kubernetes readiness probe, every pod in the deployment has just left the load
balancer because a third party had a bad minute.
--- a single component, addressed directly ---
$ curl -s http://localhost:8080/actuator/health/ordersDatabase
{
"details": {
"orders": 3,
"queryMs": 0,
"slowThresholdMs": 250
},
"status": "UP"
}
$ curl -s http://localhost:8080/actuator/health/kafka
{
"details": {
"bootstrap": "localhost:9092",
"tuned": true,
"error": "TimeoutException: null",
"probeMs": 1500,
"budgetMs": 1500
},
"status": "DOWN"
}
--- restore ---
upstream mode = UP

65
docs/output/08-groups-and-probes.txt Normal file
View File

@@ -0,0 +1,65 @@
### profile: groups
--- baseline: upstream UP ---
GET /actuator/health HTTP 503
GET /actuator/health/liveness HTTP 200
GET /actuator/health/readiness HTTP 200
GET /actuator/health/startup HTTP 200
--- upstream DOWN (a third party is having an outage) ---
GET /actuator/health HTTP 503
GET /actuator/health/liveness HTTP 200
GET /actuator/health/readiness HTTP 503
GET /actuator/health/startup HTTP 200
liveness stayed 200. readiness went 503.
Kubernetes takes this instance out of the Service and leaves the process alone.
Wire readiness to /actuator/health and you get a restart loop instead.
--- what each group contains ---
$ curl -s http://localhost:8080/actuator/health/liveness
{
"components": {
"diskSpace": {
"details": {
"total": 10213466112,
"free": 3877605376,
"threshold": 10485760,
"path": "/tmp/work/spring-boot-demo/.",
"exists": true
},
"status": "UP"
},
"livenessState": {
"status": "UP"
}
},
"status": "UP"
}
$ curl -s http://localhost:8080/actuator/health/readiness
{
"components": {
"externalApi": {
"details": {
"error": "org.springframework.web.client.HttpServerErrorException$ServiceUnavailable: 503 : \"upstream unavailable\"",
"url": "http://localhost:8080/stub/upstream/ping",
"latencyMs": 66,
"timeoutMs": 750
},
"status": "DOWN"
},
"ordersDatabase": {
"details": {
"orders": 3,
"queryMs": 0,
"slowThresholdMs": 250
},
"status": "UP"
},
"readinessState": {
"status": "UP"
}
},
"status": "DOWN"
}

15
docs/output/09-kafka-timeout.txt.naive Normal file
View File

@@ -0,0 +1,15 @@
### Kafka health check against an unreachable broker -- naive
$ time curl -s http://localhost:8080/actuator/health/kafka
{
"details": {
"bootstrap": "localhost:9092",
"tuned": false,
"error": "TimeoutException: Timed out waiting for a node assignment. Call: listNodes",
"probeMs": 60002,
"budgetMs": 180000
},
"status": "DOWN"
}
wall clock: 60.2 s

15
docs/output/09-kafka-timeout.txt.tuned Normal file
View File

@@ -0,0 +1,15 @@
### Kafka health check against an unreachable broker -- tuned
$ time curl -s http://localhost:8080/actuator/health/kafka
{
"details": {
"bootstrap": "localhost:9092",
"tuned": true,
"error": "TimeoutException: null",
"probeMs": 1500,
"budgetMs": 1500
},
"status": "DOWN"
}
wall clock: 1.6 s

18
docs/output/10-slow-upstream.txt Normal file
View File

@@ -0,0 +1,18 @@
### profile: details upstream deliberately sleeping 30s per request
### demo.upstream.timeout-ms = 750, so the indicator gives up long before the stub replies
$ time curl -s http://localhost:8080/actuator/health/externalApi
{
"details": {
"error": "org.springframework.web.client.ResourceAccessException: I/O error on GET request for \"http://localhost:8080/stub/upstream/ping\": Read timed out",
"url": "http://localhost:8080/stub/upstream/ping",
"latencyMs": 753,
"timeoutMs": 750
},
"status": "DOWN"
}
wall clock: 0.84 s
The read timeout is what bounds this, not the endpoint. Remove setReadTimeout from
ExternalApiHealthIndicator and this call blocks for the full 30 seconds, holding a
Tomcat worker the whole time.