asmhatreandClaude Sonnet 5
e478eafda3
Companion code for Zero-Downtime Database Migrations: Expand-Contract in Practice with Spring Boot: a full expand/migrate-writes/migrate-reads/contract sequence run as an actual rolling deploy across two live replicas, with a load generator sending continuous HTTP traffic through all four deploys (99.98% success, every residual error traced to a root cause rather than left unexplained). Findings include a real NOT NULL constraint trap in the expand migration, a backfill-window bug in the read switch, H2's AUTO_SERVER=TRUE single-point-of-failure behavior under a rolling restart, the drain-before-SIGTERM fix needed to close a health-check gap during graceful shutdown, and H2 silently discarding a concurrently committed INSERT during an ALTER TABLE ADD/DROP COLUMN rebuild - confirmed, by primary source, to be an H2-specific behavior rather than a property of the technique itself. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_019Fb7vW8vLyLKngBc4R3huA
2.5 KiB
10. What happens if you drop too soon
← 9. The contract migration · Next: 11. The load generator →
The second half of
ContractSafetyTest
runs Deploy 4b's migration first, and only then starts a Stage 1 instance against the
result — standing in for a canary that never got promoted, a rollback that didn't
fully take, or simply running the drop before confirming the fleet:
-- What a lingering Stage 1 instance sees if the drop runs before it is retired --
Stage 1 create() after V3 dropped "email": org.springframework.jdbc.BadSqlGrammarException
message: PreparedStatementCallback; bad SQL grammar [INSERT INTO customers(name, email) VALUES (?, ?)]
root cause: org.h2.jdbc.JdbcSQLSyntaxErrorException: Column "EMAIL" not found; SQL statement:
INSERT INTO customers(name, email) VALUES (?, ?) [42122-240]
Full transcript:
docs/output/10-drop-too-soon.txt.
This is the correct outcome, not a bug to work around. A Stage 1 instance still
running after the drop is itself the actual mistake — a deploy that didn't finish, or
a rollback nobody noticed failed — and the database telling it loudly and immediately
that email doesn't exist is far better than the alternative of silently accepting
partial writes or, worse, dropping rows. Expand-contract's safety comes from when
you're allowed to run Deploy 4b (only after confirming 100% Stage 4), not from Deploy
4b itself being forgiving of running early.
In a real deploy pipeline, this is the argument for gating Deploy 4b on an explicit
health/version check across the fleet — every instance's /actuator/info or
equivalent reporting Stage 4 — rather than a fixed timer. "Deploy 2 usually finishes
rolling out in three minutes" is not the same guarantee as "every replica confirmed
Stage 4", and the difference between them is exactly the window this chapter's test
is exploiting.
Going deeper
- Chapter 15 turns this into an actual gate: what to check, and where, before running a contract migration in a real pipeline.
← 9. The contract migration · Next: 11. The load generator →