Claude
b81af72bc3
Companion code for "Spring Batch on Boot 4.1: Jobs, Steps, Chunk Processing and Restartability". A productImportJob configured three ways by profile against a poisoned CSV row, run as real java -jar processes (not just JUnit) so the restart story is genuine: a chunk fails and rolls back, the process exits, a brand-new JVM against the same file-based H2 database resumes at the exact next unread row (READ_COUNT 20, not 60) and completes. Findings the build pins: - StepBuilder.chunk(int, PlatformTransactionManager) still compiles in Batch 6.0.5 but returns the legacy SimpleStepBuilder; chunk(int) returns the new ChunkOrientedStepBuilder, and only the latter is used here. - Two different ExecutionContext classes now exist in two different packages (infrastructure.item vs core.repository.persistence) with different shapes. - spring-boot-starter-batch alone gives a resourceless JobRepository that forgets every JobInstance the moment the JVM exits; spring-boot-starter- batch-jdbc is what makes the restart demo possible at all, demonstrated by excluding BatchJdbcAutoConfiguration and watching a "restart" collide with the previous run's own data instead of resuming it. - A migration-guide summary claiming CommandLineJobRunner was removed in 6.0 is wrong -- javap against the real jar shows @Deprecated(forRemoval=true), not removed. - RepeatStatus moved from core.repeat to infrastructure.repeat, caught by the compiler rather than by reading docs. 11 documentation chapters, 10 captured transcripts (unit tests, javap output, and real two-JVM scenario runs), all regenerated by scripts/run-all.sh. Fixed after push: three dead docs.spring.io links in the doc chapters (readersAndWriters/* and chunk-oriented-processing/*.html paths moved when Spring Batch 6 reorganized its reference docs; corrected to the current readers-and-writers/*, processor.html and chunk-oriented-processing.html paths, verified 200 via curl before committing). Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_019DXsJ1zpikbA1MQJN6RqFA
4.7 KiB
3. Chunk-oriented processing: what "chunk(10)" actually does
← Previous | README | Next: The item processor as a filter →
chunk(10) means: read up to 10 items (calling the reader once per item), run each through the
processor, then hand all the survivors to the writer in one call, inside one transaction.
If the writer succeeds, the transaction commits and the step's counters — read, write,
filter, commit — move forward together. If anything in that chunk throws, the whole
transaction rolls back: none of those 10 items' writes are kept, even the ones that would have
succeeded on their own.
The happy-path run of this module's productImportJob against clean data makes this concrete.
60 rows, chunk size 10, two rows fail the processor's validation (filtered, not written):
$ SELECT read_count, filter_count, write_count, commit_count FROM BATCH_STEP_EXECUTION;
READ_COUNT | FILTER_COUNT | WRITE_COUNT | COMMIT_COUNT
60 | 2 | 58 | 6
Full transcript: docs/output/05-happy-path.txt, source:
ProductValidatingProcessor.java.
60 rows at chunk size 10 is 6 chunks — COMMIT_COUNT confirms all 6 committed, one
transaction each.
Why 10, and what changes if you pick a different size
Chunk size is a trade-off with no universally right answer:
- Smaller chunks commit more often, so a failure loses less work and transactions are shorter (less lock contention, smaller rollback). They also mean more round trips to the database — more commit overhead per row.
- Larger chunks amortize that overhead across more rows, but a failure anywhere in the chunk costs you the whole chunk's work, and the transaction held open is bigger and longer.
10 is small enough that chapter 7's poisoned row costs you 12 rows of rework at most (one chunk), not thousands, and small enough to demonstrate multiple commits from 60 rows without a huge fixture file. Production jobs processing millions of rows commonly use chunk sizes in the hundreds or low thousands; the right number depends on row size, write cost, and how expensive a partial redo is for your specific job — there is no formula that replaces measuring it against your own writer.
Going deeper
- The chunk-processing reference: Spring Batch — Chunk-oriented processing (
rel="nofollow"). ChunkOrientedStepitself, if you want to see the commit/rollback logic this chapter describes:org.springframework.batch.core.step.item.ChunkOrientedStepinspring-batch-core-6.0.5.jar(the stack trace indocs/output/07-restart-run1-fails.txtnames it directly).- What happens when the processor itself decides to drop a row rather than the writer failing: chapter 4.