Add db-migrations-expand-contract: zero-downtime schema migrations proven with a real 4-deploy rolling run

Companion code for Zero-Downtime Database Migrations: Expand-Contract in Practice
with Spring Boot: a full expand/migrate-writes/migrate-reads/contract sequence run
as an actual rolling deploy across two live replicas, with a load generator sending
continuous HTTP traffic through all four deploys (99.98% success, every residual
error traced to a root cause rather than left unexplained). Findings include a real
NOT NULL constraint trap in the expand migration, a backfill-window bug in the read
switch, H2's AUTO_SERVER=TRUE single-point-of-failure behavior under a rolling
restart, the drain-before-SIGTERM fix needed to close a health-check gap during
graceful shutdown, and H2 silently discarding a concurrently committed INSERT during
an ALTER TABLE ADD/DROP COLUMN rebuild - confirmed, by primary source, to be an
H2-specific behavior rather than a property of the technique itself.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_019Fb7vW8vLyLKngBc4R3huA

db-migrations-expand-contract/docs/04-the-dual-write.md

@@ -0,0 +1,61 @@
+# 4. The dual write
+
+[← 3. Why migrations run outside the app](03-why-migrations-run-outside-the-app.md) · [Next: 5. The read switch →](05-the-read-switch.md)
+
+Deploy 2 is the first code deploy in the sequence, and the only thing it changes is
+what `create()` and `updateEmail()` write. Stage 2's SQL in
+[`CustomerService`](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/db-migrations-expand-contract/src/main/java/com/ankurm/expandcontract/customer/CustomerService.java)
+writes to both columns:
+
+```java
+case 2, 3 -> jdbc.sql("INSERT INTO customers(name, email, email_address) VALUES (?, ?, ?)")
+        .param(name).param(email).param(email)
+        .update(keyHolder, "id");
+```
+
+```java
+case 2, 3 -> jdbc.sql("UPDATE customers SET email = ?, email_address = ? WHERE id = ?")
+        .param(newEmail).param(newEmail).param(id).update();
+```
+
+Stage 2 and Stage 3 share this write path — the only difference between them is what
+they *read*, covered in [chapter 5](05-the-read-switch.md). That's deliberate: writes
+have to stay dual for two whole deploys (2 and 3) so that by the time Deploy 4
+arrives, every row in the table — regardless of which stage wrote it last — is
+guaranteed to have both columns populated identically.
+
+[`DualWriteConsistencyTest`](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/db-migrations-expand-contract/src/test/java/com/ankurm/expandcontract/DualWriteConsistencyTest.java)
+confirms both halves of that promise — a fresh create lands in both columns, and an
+update *replaces* the value in both, not just one:
+
+```
+-- after create() --
+NAME              | EMAIL                 | EMAIL_ADDRESS
+------------------+-----------------------+----------------------
+Margaret Hamilton | [email protected] | [email protected]
+
+-- after updateEmail() - the old value is gone from BOTH columns, not just one --
+NAME              | EMAIL                   | EMAIL_ADDRESS
+------------------+-------------------------+------------------------
+Margaret Hamilton | [email protected] | [email protected]
+```
+
+Full transcript:
+[`docs/output/04-dual-write-consistency.txt`](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/db-migrations-expand-contract/docs/output/04-dual-write-consistency.txt).
+
+The second half of that test matters more than it looks. A dual write that only
+inserts into both columns but updates only one is a much more common bug than it
+sounds — the update path is usually written later, by someone who's already stopped
+thinking about `email_address` because the create path "already handles the new
+column".
+
+## Going deeper
+
+- Deploy 2's own rollout window — where some replicas are still Stage 1 while others
+  are already Stage 2 — is exactly the gap [chapter 7](07-the-backfill-window-bug.md)
+  is about: what a lingering Stage 1 write during *this* rollout means for the read
+  switch that comes next.
+- The full cross-stage read/write matrix, including this deploy's pair, is proven
+  directly in [chapter 8](08-the-rolling-window-proof.md).
+
+[← 3. Why migrations run outside the app](03-why-migrations-run-outside-the-app.md) · [Next: 5. The read switch →](05-the-read-switch.md)