spring-boot-demo/docs/07-groups-and-probes.md

← 06 Failure modes · 07 · Groups, probes and Kubernetes · 08 The diagnostics endpoint →

07 — Groups, probes and Kubernetes

The failure this prevents

A third-party API goes down. Your externalApi indicator reports DOWN. /actuator/health aggregates to DOWN and answers 503. Your Kubernetes manifest points both probes at it:

livenessProbe:  { httpGet: { path: /actuator/health, port: 8080 } }
readinessProbe: { httpGet: { path: /actuator/health, port: 8080 } }

The readiness failure is correct — this instance cannot serve. The liveness failure is a catastrophe: the kubelet kills the container, every replica fails the same probe at the same time, and the whole deployment enters a restart loop. A partial outage in someone else's system has become a total outage in yours, and the restarts make recovery slower because every fresh JVM has to warm up while the upstream is still failing.

Liveness answers "is this process broken beyond recovery?" Almost nothing external belongs in it, because restarting cannot fix an external problem.

Readiness answers "should this instance receive traffic right now?" Dependencies belong here.

The configuration

From application-groups.yaml:

management:
  endpoint:
    health:
      group:
        liveness:
          include: livenessState,diskSpace
        readiness:
          include: readinessState,ordersDatabase,externalApi
        startup:
          include: ordersDatabase
      probes:
        enabled: true

Each group gets its own path: /actuator/health/liveness, /actuator/health/readiness, /actuator/health/startup. Groups can carry their own show-details too.

The evidence

output/08-groups-and-probes.txt:

--- baseline: upstream UP ---
  GET /actuator/health              HTTP 503
  GET /actuator/health/liveness     HTTP 200
  GET /actuator/health/readiness    HTTP 200
  GET /actuator/health/startup      HTTP 200

--- upstream DOWN ---
  GET /actuator/health              HTTP 503
  GET /actuator/health/liveness     HTTP 200
  GET /actuator/health/readiness    HTTP 503
  GET /actuator/health/startup      HTTP 200

Liveness stayed 200 across the outage. Readiness moved. Kubernetes removes this instance from the Service endpoints and leaves the process alone; when the upstream recovers, readiness goes green and traffic returns without a single restart.

Note the aggregate /actuator/health was 503 in both columns — the Kafka indicator is down throughout, since this repository runs no broker. That is exactly why you should not point a probe at the aggregate: it is the union of everything, and it tells the orchestrator nothing actionable.

The manifest

livenessProbe:
  httpGet: { path: /actuator/health/liveness, port: 9001 }
  periodSeconds: 10
  failureThreshold: 3
readinessProbe:
  httpGet: { path: /actuator/health/readiness, port: 9001 }
  periodSeconds: 5
  failureThreshold: 2
startupProbe:
  httpGet: { path: /actuator/health/startup, port: 9001 }
  periodSeconds: 5
  failureThreshold: 30

Port 9001 is the management port from chapter 04. The kubelet reaches it inside the pod network; the ingress does not.

The startupProbe matters more than people think. Without one, a JVM that takes 45 seconds to warm up under failureThreshold: 3 and periodSeconds: 10 gets killed at 30 seconds, forever, and the symptom is a crash-loop with no error in the logs.

readinessState, and shutting down cleanly

readinessState is Boot's own indicator, driven by ApplicationAvailability. During graceful shutdown Boot flips it to REFUSING_TRAFFIC before the server stops accepting, so readiness goes red while in-flight requests finish. Pair it with:

server:
  shutdown: graceful
spring:
  lifecycle:
    timeout-per-shutdown-phase: 30s

You can also drive it yourself by publishing an AvailabilityChangeEvent — useful for taking an instance out of rotation before a risky migration.

Which dependencies belong in readiness

Ask: if this is down, can this instance still serve any useful request?

• Primary database → yes, readiness.
• Kafka, for a service whose only job is consuming → yes, readiness.
• Kafka, for a service that also serves reads → probably not; degrade rather than withdraw.
• A recommendations API you fall back to a static list for → no. Do not check it at all.

Every dependency in readiness is a dependency that can take your service out of rotation. That list should be shorter than your instinct suggests.

---

← 06 · 07 · 08 The diagnostics endpoint →