Add db-migrations-expand-contract: zero-downtime schema migrations proven with a real 4-deploy rolling run

Companion code for Zero-Downtime Database Migrations: Expand-Contract in Practice
with Spring Boot: a full expand/migrate-writes/migrate-reads/contract sequence run
as an actual rolling deploy across two live replicas, with a load generator sending
continuous HTTP traffic through all four deploys (99.98% success, every residual
error traced to a root cause rather than left unexplained). Findings include a real
NOT NULL constraint trap in the expand migration, a backfill-window bug in the read
switch, H2's AUTO_SERVER=TRUE single-point-of-failure behavior under a rolling
restart, the drain-before-SIGTERM fix needed to close a health-check gap during
graceful shutdown, and H2 silently discarding a concurrently committed INSERT during
an ALTER TABLE ADD/DROP COLUMN rebuild - confirmed, by primary source, to be an
H2-specific behavior rather than a property of the technique itself.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_019Fb7vW8vLyLKngBc4R3huA
This commit is contained in:
2026-09-16 19:22:48 +00:00
co-authored by Claude Sonnet 5
parent 22c30d4a5b
commit e478eafda3
60 changed files with 3561 additions and 0 deletions
@@ -0,0 +1,36 @@
=========================================================================
Deploy 1 (EXPAND): additive column + backfill, Stage 1 code untouched
=========================================================================
captured: 2026-09-16T19:20:56.676919361Z
-- schema before Deploy 1 --
COLUMN_NAME
-----------
ID
NAME
EMAIL
CREATED_AT
(4 rows)
-- schema after Deploy 1 (email_address added) --
COLUMN_NAME
-------------
ID
NAME
EMAIL
CREATED_AT
EMAIL_ADDRESS
(5 rows)
-- Ada's row was backfilled by the migration itself --
NAME | EMAIL | EMAIL_ADDRESS
-------------+------------------+-----------------
Ada Lovelace | [email protected] | [email protected]
(1 row)
-- Stage 1's original INSERT still works, unmodified, after the migration --
NAME | EMAIL | EMAIL_ADDRESS
-------------+--------------------+--------------
Grace Hopper | [email protected] | NULL
(1 row)