Add db-migrations-expand-contract: zero-downtime schema migrations proven with a real 4-deploy rolling run

Companion code for Zero-Downtime Database Migrations: Expand-Contract in Practice with Spring Boot: a full expand/migrate-writes/migrate-reads/contract sequence run as an actual rolling deploy across two live replicas, with a load generator sending continuous HTTP traffic through all four deploys (99.98% success, every residual error traced to a root cause rather than left unexplained). Findings include a real NOT NULL constraint trap in the expand migration, a backfill-window bug in the read switch, H2's AUTO_SERVER=TRUE single-point-of-failure behavior under a rolling restart, the drain-before-SIGTERM fix needed to close a health-check gap during graceful shutdown, and H2 silently discarding a concurrently committed INSERT during an ALTER TABLE ADD/DROP COLUMN rebuild - confirmed, by primary source, to be an H2-specific behavior rather than a property of the technique itself. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_019Fb7vW8vLyLKngBc4R3huA
2026-09-16 19:22:48 +00:00
parent 22c30d4a5b
commit e478eafda3
60 changed files with 3561 additions and 0 deletions
@@ -0,0 +1,47 @@
+# 9. The contract migration
+
+[← 8. The rolling window proof](08-the-rolling-window-proof.md) · [Next: 10. What happens if you drop too soon →](10-what-happens-if-you-drop-too-soon.md)
+
+Deploy 4 is split into two parts on purpose, the same way Deploy 1 was schema-only
+and Deploys 2/3 were code-only:
+
+- **4a — code.** A rolling restart to Stage 4, which never reads or writes `email`.
+  The old column is still physically present; Stage 4 code simply ignores it.
+- **4b — schema.** Once every replica is confirmed on Stage 4 code, and only then,
+  [`V3__drop_email_column.sql`](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/db-migrations-expand-contract/src/main/resources/db/migration/V3__drop_email_column.sql)
+  runs, with zero app restarts:
+
+```sql
+ALTER TABLE customers DROP COLUMN email;
+```
+
+[`ContractSafetyTest`](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/db-migrations-expand-contract/src/test/java/com/ankurm/expandcontract/ContractSafetyTest.java)
+confirms the happy path: Stage 4 reads and writes keep working, both for a row that
+existed before the drop and for one created entirely after it —
+
+```
+-- Stage 4 read, after V3 dropped the email column --
+Customer[id=1, name=Annie Easley, [email protected]]
+
+-- Stage 4 create + read, entirely after the drop --
+Customer[id=2, name=Mary Allen Wilkes, [email protected]]
+```
+
+Full transcript:
+[`docs/output/09-contract-safety.txt`](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/db-migrations-expand-contract/docs/output/09-contract-safety.txt).
+
+Why not drop the column in the same deploy as the code change? Because "every
+replica is confirmed on Stage 4" is a statement about the *fleet*, and a rolling
+deploy makes it true only once, at the very end of the rollout — never at the moment
+the deploy starts. Running 4b before that point is dropping a column a live Stage 1,
+2, or 3 replica might still need. What that actually looks like, captured directly,
+is [chapter 10](10-what-happens-if-you-drop-too-soon.md).
+
+## Going deeper
+
+- The DDL Deploy 4b runs is not, on this specific database engine, quite as free of
+  side effects on concurrent traffic as "just a schema change" suggests — see
+  [chapter 14](14-the-ddl-lock-window.md) for what H2 actually does while this
+  statement executes.
+
+[← 8. The rolling window proof](08-the-rolling-window-proof.md) · [Next: 10. What happens if you drop too soon →](10-what-happens-if-you-drop-too-soon.md)