Add db-migrations-expand-contract: zero-downtime schema migrations proven with a real 4-deploy rolling run
Companion code for Zero-Downtime Database Migrations: Expand-Contract in Practice with Spring Boot: a full expand/migrate-writes/migrate-reads/contract sequence run as an actual rolling deploy across two live replicas, with a load generator sending continuous HTTP traffic through all four deploys (99.98% success, every residual error traced to a root cause rather than left unexplained). Findings include a real NOT NULL constraint trap in the expand migration, a backfill-window bug in the read switch, H2's AUTO_SERVER=TRUE single-point-of-failure behavior under a rolling restart, the drain-before-SIGTERM fix needed to close a health-check gap during graceful shutdown, and H2 silently discarding a concurrently committed INSERT during an ALTER TABLE ADD/DROP COLUMN rebuild - confirmed, by primary source, to be an H2-specific behavior rather than a property of the technique itself. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_019Fb7vW8vLyLKngBc4R3huA
This commit is contained in:
+23
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env bash
|
||||
# Starts one replica on a given port and stage, and waits for it to report healthy.
|
||||
# Usage: start-instance.sh <port> <stage>
|
||||
set -euo pipefail
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
source "$DIR/env.sh"
|
||||
|
||||
PORT="$1"
|
||||
STAGE="$2"
|
||||
|
||||
# -XX:TieredStopAtLevel=1 and a small fixed heap cut this JVM's own startup CPU
|
||||
# burst dramatically (C2 compilation and heap sizing are the two biggest costs of a
|
||||
# cold Spring Boot start). That matters here specifically because two replicas share
|
||||
# a 2-core sandbox: a slow-starting replica can starve its own sibling's health
|
||||
# checks long enough to look like an outage that never actually happened - see
|
||||
# docs/11-the-load-generator.md.
|
||||
nohup java -XX:TieredStopAtLevel=1 -XX:+UseSerialGC -Xms128m -Xmx256m \
|
||||
-jar "$JAR" --server.port="$PORT" --app.stage="$STAGE" \
|
||||
> "$EC_LOG_DIR/instance-$PORT.log" 2>&1 < /dev/null &
|
||||
echo $! > "$EC_PID_DIR/$PORT.pid"
|
||||
echo "started stage $STAGE on port $PORT (pid $(cat "$EC_PID_DIR/$PORT.pid"))"
|
||||
wait_healthy "$PORT"
|
||||
echo "port $PORT healthy"
|
||||
Reference in New Issue
Block a user