Ankur Mhatre
4b6cefa60a
Companion repository for the ankurm.com article. Every transcript in docs/output/ was produced by running this project; scripts/run-all.sh regenerates all of them. Verified against Spring Boot 4.1.1 / Framework 7.0.9 / Security 7.1.1 / Micrometer 1.17.1 / kafka-clients 4.2.1 on Temurin JDK 25.0.4.1+1.
113 lines
4.5 KiB
Markdown
113 lines
4.5 KiB
Markdown
[← 04 Securing Actuator](04-securing-actuator.md) · **05 · Custom health indicators** · [06 Failure modes →](06-health-indicator-failure-modes.md)
|
|
|
|
# 05 — Custom health indicators
|
|
|
|
Three indicators, three different jobs.
|
|
|
|
| Bean | Class | Checks |
|
|
|---|---|---|
|
|
| `ordersDatabase` | [`OrdersDatabaseHealthIndicator`](../src/main/java/com/ankurm/actuator/health/OrdersDatabaseHealthIndicator.java) | the query the application depends on |
|
|
| `kafka` | [`KafkaHealthIndicator`](../src/main/java/com/ankurm/actuator/health/KafkaHealthIndicator.java) | `AdminClient.describeCluster` |
|
|
| `externalApi` | [`ExternalApiHealthIndicator`](../src/main/java/com/ankurm/actuator/health/ExternalApiHealthIndicator.java) | an HTTP dependency, with a budget |
|
|
|
|
## The shape
|
|
|
|
```java
|
|
import org.springframework.boot.health.contributor.AbstractHealthIndicator;
|
|
import org.springframework.boot.health.contributor.Health;
|
|
|
|
@Component("ordersDatabase")
|
|
public class OrdersDatabaseHealthIndicator extends AbstractHealthIndicator {
|
|
|
|
@Override
|
|
protected void doHealthCheck(Health.Builder builder) throws Exception {
|
|
// throw, or call builder.up() / down() / status("DEGRADED")
|
|
}
|
|
}
|
|
```
|
|
|
|
Note the package — `org.springframework.boot.health.contributor`, new in Boot 4. See
|
|
[chapter 02](02-boot-4-changes.md).
|
|
|
|
`AbstractHealthIndicator` over the bare interface, because it catches your exceptions and turns
|
|
them into `DOWN` with the error recorded, rather than letting one throwing indicator take down
|
|
the whole `/actuator/health` response.
|
|
|
|
## The bean name is the JSON key
|
|
|
|
`@Component("ordersDatabase")` produces `"ordersDatabase"` in the response. Boot strips a
|
|
trailing `HealthIndicator` from the bean name, so `ordersDatabaseHealthIndicator` gives the same
|
|
key. Rename the bean and every dashboard, alert and health-group `include:` that referenced the
|
|
old key silently stops matching — which is why
|
|
[`HealthGroupTests`](../src/test/java/com/ankurm/actuator/HealthGroupTests.java) asserts on the
|
|
names.
|
|
|
|
## Check what the application needs, not what the connection can do
|
|
|
|
Boot's built-in `db` indicator runs a validation query and answers "can I reach the database".
|
|
That is rarely the question. `ordersDatabase` runs the query the service actually depends on and
|
|
adds a latency judgement:
|
|
|
|
```json
|
|
{"details":{"orders":3,"queryMs":0,"slowThresholdMs":250},"status":"UP"}
|
|
```
|
|
|
|
It returns a custom `DEGRADED` status above the threshold. Custom statuses are legal;
|
|
`management.endpoint.health.status.order` and `.http-mapping` control how they aggregate and
|
|
what HTTP code they produce. An unmapped custom status aggregates as `UNKNOWN` and maps to 200,
|
|
which is usually not what you meant.
|
|
|
|
## Every indicator needs a budget
|
|
|
|
All three carry an explicit timeout, and the reason is [chapter 06](06-health-indicator-failure-modes.md).
|
|
|
|
- `ordersDatabase` — `jdbc.setQueryTimeout(2)`
|
|
- `externalApi` — connect and read timeouts of 750 ms on the `RestClient` request factory
|
|
- `kafka` — four separate Kafka bounds plus an outer `KafkaFuture.get(timeout)`
|
|
|
|
## Credentials
|
|
|
|
`externalApi` sends HTTP Basic. The first captured run of this repository did not, and reported:
|
|
|
|
```json
|
|
{"error":"HttpClientErrorException$Unauthorized: 401 : [no body]","status":"DOWN"}
|
|
```
|
|
|
|
which looks exactly like an upstream outage and was a missing header. If your indicator
|
|
authenticates, make the failure message distinguish "they are down" from "we are unauthorised" —
|
|
future you will be reading it at 3am.
|
|
|
|
## Reuse the client
|
|
|
|
`KafkaHealthIndicator` creates one `AdminClient` in its constructor and closes it in
|
|
`close()`. Creating one per probe opens a fresh set of broker connections on every probe
|
|
interval; across a fleet of any size that is a denial-of-service against your own brokers.
|
|
|
|
## What it all looks like
|
|
|
|
[`output/07-custom-health-indicators.txt`](output/07-custom-health-indicators.txt) has the full
|
|
nine-component response. Flipping the stub upstream to `DOWN` and changing nothing else:
|
|
|
|
```json
|
|
{
|
|
"status": "DOWN",
|
|
"externalApi": {
|
|
"details": {
|
|
"error": "HttpServerErrorException$ServiceUnavailable: 503 : \"upstream unavailable\"",
|
|
"url": "http://localhost:8080/stub/upstream/ping",
|
|
"latencyMs": 101,
|
|
"timeoutMs": 750
|
|
},
|
|
"status": "DOWN"
|
|
}
|
|
}
|
|
```
|
|
|
|
`/actuator/health` now answers 503. If that URL is your readiness probe, every pod in the
|
|
deployment just left the load balancer because a third party had a bad minute.
|
|
[Chapter 07](07-groups-and-probes.md) fixes that.
|
|
|
|
---
|
|
|
|
[← 04](04-securing-actuator.md) · **05** · [06 Failure modes →](06-health-indicator-failure-modes.md)
|