asmhatre/spring-boot-demo
1
0
Fork 0
Code Activity

Add spring-batch-partitioning: manager/worker partitioning, gridSize semantics, rejected-partition recovery, and a real 10M-row scaling sweep

Browse Source
This commit is contained in:
asmhatre
2026-09-14 09:36:35 +00:00
parent b81af72bc3
commit 34e9f5b243
40 changed files with 2071 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
spring-batch-partitioning/docs/output/05-happy-path-4-partitions.txt
+65
View File
@@ -0,0 +1,65 @@
# Happy path: 4 shard files, gridSize 4, pool size 4 -- 20,000 rows
$ java -jar target/spring-batch-partitioning-1.0.0.jar \
--partition.shards-dir=./data/shards-small --partition.grid-size=4 \
--partition.pool-core-size=4 --partition.pool-max-size=4
2026-09-14T09:29:56.758Z INFO 4937 --- [ main] c.a.b.PartitioningDemoApplication : Started PartitioningDemoApplication in 3.563 seconds (process running for 4.114)
2026-09-14T09:29:56.876Z INFO 4937 --- [ main] o.s.batch.core.step.AbstractStep : Executing step: [ordersManagerStep]
2026-09-14T09:29:56.892Z INFO 4937 --- [der-partition-2] o.s.batch.core.step.AbstractStep : Executing step: [ordersWorkerStep:partition0]
2026-09-14T09:29:56.890Z INFO 4937 --- [der-partition-1] o.s.batch.core.step.AbstractStep : Executing step: [ordersWorkerStep:partition1]
2026-09-14T09:29:56.900Z INFO 4937 --- [der-partition-4] o.s.batch.core.step.AbstractStep : Executing step: [ordersWorkerStep:partition2]
2026-09-14T09:29:56.901Z INFO 4937 --- [der-partition-3] o.s.batch.core.step.AbstractStep : Executing step: [ordersWorkerStep:partition3]
2026-09-14T09:29:57.962Z INFO 4937 --- [der-partition-2] o.s.batch.core.step.AbstractStep : Step: [ordersWorkerStep:partition0] executed in 1s70ms
2026-09-14T09:29:57.963Z INFO 4937 --- [der-partition-4] o.s.batch.core.step.AbstractStep : Step: [ordersWorkerStep:partition2] executed in 1s64ms
2026-09-14T09:29:57.990Z INFO 4937 --- [der-partition-1] o.s.batch.core.step.AbstractStep : Step: [ordersWorkerStep:partition1] executed in 1s99ms
2026-09-14T09:29:57.998Z INFO 4937 --- [der-partition-3] o.s.batch.core.step.AbstractStep : Step: [ordersWorkerStep:partition3] executed in 1s98ms
2026-09-14T09:29:58.005Z INFO 4937 --- [ main] o.s.batch.core.step.AbstractStep : Step: [ordersManagerStep] executed in 1s130ms
2026-09-14T09:29:58.011Z INFO 4937 --- [ main] o.s.batch.core.step.AbstractStep : Executing step: [reportStep]
REPORT: 20000 orders scored, 1561 flagged high-risk
JOB FINISHED: id=1 status=COMPLETED exitCode=COMPLETED
All four partitions start within a few milliseconds of each other, on four distinct named
threads (order-partition-1..4) -- this is what "partitioned" actually looks like at the OS level,
not just in configuration. 1,561 of 20,000 orders (7.8%) came back flagged high-risk, matching the
expected rate from the threshold math in docs/06-why-cpu-bound-not-io-bound.md.
--- GET /batch/partitions/1 (the diagnostic endpoint -- delete before shipping) ---
[
{
"PARTITION_NAME": "ordersWorkerStep:partition0",
"THREAD_NAME": "order-partition-2",
"READ_COUNT": 5000,
"STARTED_AT": "2026-09-14T09:29:56.892Z",
"FINISHED_AT": "2026-09-14T09:29:57.968Z",
"DURATION_MS": 1075,
"EXIT_CODE": "COMPLETED"
},
{
"PARTITION_NAME": "ordersWorkerStep:partition1",
"THREAD_NAME": "order-partition-1",
"READ_COUNT": 5000,
"STARTED_AT": "2026-09-14T09:29:56.895Z",
"FINISHED_AT": "2026-09-14T09:29:57.999Z",
"DURATION_MS": 1103,
"EXIT_CODE": "COMPLETED"
},
{
"PARTITION_NAME": "ordersWorkerStep:partition2",
"THREAD_NAME": "order-partition-4",
"READ_COUNT": 5000,
"STARTED_AT": "2026-09-14T09:29:56.900Z",
"FINISHED_AT": "2026-09-14T09:29:57.964Z",
"DURATION_MS": 1064,
"EXIT_CODE": "COMPLETED"
},
{
"PARTITION_NAME": "ordersWorkerStep:partition3",
"THREAD_NAME": "order-partition-3",
"READ_COUNT": 5000,
"STARTED_AT": "2026-09-14T09:29:56.901Z",
"FINISHED_AT": "2026-09-14T09:29:58.002Z",
"DURATION_MS": 1101,
"EXIT_CODE": "COMPLETED"
}
]