# 13. Graceful shutdown vs. kill -9

[← 12. The AUTO_SERVER trap](12-the-auto-server-trap.md) · [Next: 14. The DDL lock window →](14-the-ddl-lock-window.md)

`application.yml` sets one line that does nothing by itself:

```yaml
server:
  shutdown: graceful
```

`server.shutdown: graceful` only changes behavior on `SIGTERM` — it stops accepting
new connections but lets in-flight requests finish before the process exits. An
earlier version of
[`stop-instance.sh`](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/db-migrations-expand-contract/scripts/stop-instance.sh)
used `kill -9`, which bypasses graceful shutdown entirely — the process disappears
mid-request, and every request in flight at that instant surfaces in the load
generator as a raw `ConnectException` or `IOException`. Switching to `SIGTERM`
(`kill -15`), with a bounded wait for a clean exit and `SIGKILL` only as a fallback,
is the first half of the fix:

```bash
kill -15 "$PID"
for i in $(seq 1 40); do
    kill -0 "$PID" 2>/dev/null || break
    sleep 0.25
done
if kill -0 "$PID" 2>/dev/null; then
    kill -9 "$PID"
fi
```

That alone wasn't enough. `server.shutdown: graceful` starts refusing new connections
the instant `SIGTERM` arrives — but the load generator's health checker polls every
300ms, and the pool didn't yet know to stop routing traffic there. The gap between
"the process just stopped accepting connections" and "the load balancer's health
check has noticed and rerouted" is exactly where `ConnectException` bursts kept
showing up, even with `SIGTERM` in place.

The second half of the fix is a way to say "stop sending me traffic" *before* the
process is touched at all:
[`DrainController`](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/db-migrations-expand-contract/src/main/java/com/ankurm/expandcontract/diag/DrainController.java):

```java
@PostMapping("/admin/drain")
public String drain() {
    AvailabilityChangeEvent.publish(events, this, ReadinessState.REFUSING_TRAFFIC);
    return "draining";
}
```

Publishing `ReadinessState.REFUSING_TRAFFIC` flips `/actuator/health`'s readiness
group immediately — this is the same event a Kubernetes-style `preStop` hook
publishes before the container is sent `SIGTERM`. `stop-instance.sh` calls it, sleeps,
*then* sends `SIGTERM`:

```bash
curl -s -X POST "http://localhost:$PORT/admin/drain" -o /dev/null || true
sleep 1.5
kill -15 "$PID"
```

That 1.5-second pause is deliberate slack for the health checker's 300ms poll
interval — enough for at least a couple of checks to land and pull this instance out
of rotation before it's asked to stop at all. Together, drain-then-SIGTERM is what
took the rolling-restart portion of the article's live run to zero
`ConnectException`/`IOException` errors — the six errors that remain in the final
summary are a completely different, database-level cause, covered next.

## Going deeper

- Spring's own `ReadinessState` and the Kubernetes probe pattern it mirrors:
  [Spring Boot reference docs, Application Availability](https://docs.spring.io/spring-boot/reference/actuator/application-availability.html) (nofollow).
- `/admin/drain` is a diagnostic-grade endpoint with no auth — see
  [chapter 15](15-production-checklist.md) for what to do with it before shipping.

[← 12. The AUTO_SERVER trap](12-the-auto-server-trap.md) · [Next: 14. The DDL lock window →](14-the-ddl-lock-window.md)