asmhatreandClaude Sonnet 5
e478eafda3
Companion code for Zero-Downtime Database Migrations: Expand-Contract in Practice with Spring Boot: a full expand/migrate-writes/migrate-reads/contract sequence run as an actual rolling deploy across two live replicas, with a load generator sending continuous HTTP traffic through all four deploys (99.98% success, every residual error traced to a root cause rather than left unexplained). Findings include a real NOT NULL constraint trap in the expand migration, a backfill-window bug in the read switch, H2's AUTO_SERVER=TRUE single-point-of-failure behavior under a rolling restart, the drain-before-SIGTERM fix needed to close a health-check gap during graceful shutdown, and H2 silently discarding a concurrently committed INSERT during an ALTER TABLE ADD/DROP COLUMN rebuild - confirmed, by primary source, to be an H2-specific behavior rather than a property of the technique itself. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_019Fb7vW8vLyLKngBc4R3huA
43 lines
2.5 KiB
Markdown
43 lines
2.5 KiB
Markdown
# 10. What happens if you drop too soon
|
|
|
|
[← 9. The contract migration](09-the-contract-migration.md) · [Next: 11. The load generator →](11-the-load-generator.md)
|
|
|
|
The second half of
|
|
[`ContractSafetyTest`](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/db-migrations-expand-contract/src/test/java/com/ankurm/expandcontract/ContractSafetyTest.java)
|
|
runs Deploy 4b's migration first, and only then starts a Stage 1 instance against the
|
|
result — standing in for a canary that never got promoted, a rollback that didn't
|
|
fully take, or simply running the drop before confirming the fleet:
|
|
|
|
```
|
|
-- What a lingering Stage 1 instance sees if the drop runs before it is retired --
|
|
Stage 1 create() after V3 dropped "email": org.springframework.jdbc.BadSqlGrammarException
|
|
message: PreparedStatementCallback; bad SQL grammar [INSERT INTO customers(name, email) VALUES (?, ?)]
|
|
root cause: org.h2.jdbc.JdbcSQLSyntaxErrorException: Column "EMAIL" not found; SQL statement:
|
|
INSERT INTO customers(name, email) VALUES (?, ?) [42122-240]
|
|
```
|
|
|
|
Full transcript:
|
|
[`docs/output/10-drop-too-soon.txt`](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/db-migrations-expand-contract/docs/output/10-drop-too-soon.txt).
|
|
|
|
This is the correct outcome, not a bug to work around. A Stage 1 instance still
|
|
running after the drop is itself the actual mistake — a deploy that didn't finish, or
|
|
a rollback nobody noticed failed — and the database telling it loudly and immediately
|
|
that `email` doesn't exist is far better than the alternative of silently accepting
|
|
partial writes or, worse, dropping rows. Expand-contract's safety comes from **when**
|
|
you're allowed to run Deploy 4b (only after confirming 100% Stage 4), not from Deploy
|
|
4b itself being forgiving of running early.
|
|
|
|
In a real deploy pipeline, this is the argument for gating Deploy 4b on an explicit
|
|
health/version check across the fleet — every instance's `/actuator/info` or
|
|
equivalent reporting Stage 4 — rather than a fixed timer. "Deploy 2 usually finishes
|
|
rolling out in three minutes" is not the same guarantee as "every replica confirmed
|
|
Stage 4", and the difference between them is exactly the window this chapter's test
|
|
is exploiting.
|
|
|
|
## Going deeper
|
|
|
|
- [Chapter 15](15-production-checklist.md) turns this into an actual gate: what to
|
|
check, and where, before running a contract migration in a real pipeline.
|
|
|
|
[← 9. The contract migration](09-the-contract-migration.md) · [Next: 11. The load generator →](11-the-load-generator.md)
|