Claude
b81af72bc3
Companion code for "Spring Batch on Boot 4.1: Jobs, Steps, Chunk Processing and Restartability". A productImportJob configured three ways by profile against a poisoned CSV row, run as real java -jar processes (not just JUnit) so the restart story is genuine: a chunk fails and rolls back, the process exits, a brand-new JVM against the same file-based H2 database resumes at the exact next unread row (READ_COUNT 20, not 60) and completes. Findings the build pins: - StepBuilder.chunk(int, PlatformTransactionManager) still compiles in Batch 6.0.5 but returns the legacy SimpleStepBuilder; chunk(int) returns the new ChunkOrientedStepBuilder, and only the latter is used here. - Two different ExecutionContext classes now exist in two different packages (infrastructure.item vs core.repository.persistence) with different shapes. - spring-boot-starter-batch alone gives a resourceless JobRepository that forgets every JobInstance the moment the JVM exits; spring-boot-starter- batch-jdbc is what makes the restart demo possible at all, demonstrated by excluding BatchJdbcAutoConfiguration and watching a "restart" collide with the previous run's own data instead of resuming it. - A migration-guide summary claiming CommandLineJobRunner was removed in 6.0 is wrong -- javap against the real jar shows @Deprecated(forRemoval=true), not removed. - RepeatStatus moved from core.repeat to infrastructure.repeat, caught by the compiler rather than by reading docs. 11 documentation chapters, 10 captured transcripts (unit tests, javap output, and real two-JVM scenario runs), all regenerated by scripts/run-all.sh. Fixed after push: three dead docs.spring.io links in the doc chapters (readersAndWriters/* and chunk-oriented-processing/*.html paths moved when Spring Batch 6 reorganized its reference docs; corrected to the current readers-and-writers/*, processor.html and chunk-oriented-processing.html paths, verified 200 via curl before committing). Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_019DXsJ1zpikbA1MQJN6RqFA
87 lines
5.4 KiB
Markdown
87 lines
5.4 KiB
Markdown
# 1. The problem, and the smallest correct mental model
|
|
|
|
[← README](../README.md) | [Next: Anatomy of a job →](02-anatomy-of-a-job.md)
|
|
|
|
A batch job processes a lot of records without a person watching each one: importing a CSV of
|
|
products into a database, closing out a day's transactions, re-indexing a search table. Two
|
|
things make this harder than writing a `for` loop over the rows:
|
|
|
|
- **It has to survive being interrupted.** A process gets killed, a database connection drops,
|
|
a row three-quarters of the way through is bad. A `for` loop that dies at row 4,700,000 has no
|
|
idea it already wrote 4,699,999 rows, and re-running it from the top either duplicates work or
|
|
duplicates data.
|
|
- **It has to report what happened**, precisely: how many rows were read, how many were written,
|
|
how many were skipped and why, whether it finished. "The import ran" is not an answer anyone
|
|
building on top of this can use.
|
|
|
|
Spring Batch's job is to own both of those problems so your code only has to describe three
|
|
things: where the rows come from, what to do to each one, and where they go. Everything else
|
|
— tracking progress, committing in batches, remembering where a failed run stopped —
|
|
is the framework's job, provided you tell it enough for it to do that job. Most of this article
|
|
is about that "provided."
|
|
|
|
## The five nouns
|
|
|
|
<figure>
|
|
<svg viewBox="0 0 740 300" role="img" aria-label="A Job contains an ordered list of Steps; each Step is chunk-oriented and reads from a reader, transforms with a processor, and writes with a writer; a JobRepository records everything happening.">
|
|
<style>
|
|
.t{font:600 14px sans-serif;fill:#1a1a1a}.h{font:600 12px sans-serif;fill:#1a1a1a}.c{font:11px sans-serif;fill:#4b5563}.m{font:11px monospace;fill:#1a1a1a}
|
|
</style>
|
|
<rect x="10" y="10" width="720" height="80" rx="6" fill="#e8eefc" stroke="#5b7fc7"/>
|
|
<text x="24" y="30" class="h">Job "productImportJob"</text>
|
|
<text x="24" y="50" class="m">start(importStep).next(reportStep)</text>
|
|
<rect x="30" y="60" width="180" height="24" rx="4" fill="#fff" stroke="#5b7fc7"/>
|
|
<text x="45" y="76" class="m">Step: importStep</text>
|
|
<rect x="230" y="60" width="180" height="24" rx="4" fill="#fff" stroke="#5b7fc7"/>
|
|
<text x="245" y="76" class="m">Step: reportStep</text>
|
|
<rect x="60" y="120" width="620" height="90" rx="6" fill="#e7f4ea" stroke="#4a9d63"/>
|
|
<text x="74" y="140" class="h">importStep is chunk-oriented</text>
|
|
<rect x="80" y="155" width="150" height="40" rx="4" fill="#fff" stroke="#4a9d63"/>
|
|
<text x="95" y="179" class="m">ItemReader</text>
|
|
<rect x="290" y="155" width="150" height="40" rx="4" fill="#fff" stroke="#4a9d63"/>
|
|
<text x="305" y="179" class="m">ItemProcessor</text>
|
|
<rect x="500" y="155" width="150" height="40" rx="4" fill="#fff" stroke="#4a9d63"/>
|
|
<text x="515" y="179" class="m">ItemWriter</text>
|
|
<line x1="230" y1="175" x2="290" y2="175" stroke="#4a9d63" marker-end="url(#arrow)"/>
|
|
<line x1="440" y1="175" x2="500" y2="175" stroke="#4a9d63" marker-end="url(#arrow)"/>
|
|
<defs><marker id="arrow" markerWidth="8" markerHeight="8" refX="6" refY="4" orient="auto"><path d="M0,0 L8,4 L0,8 z" fill="#4a9d63"/></marker></defs>
|
|
<rect x="230" y="240" width="280" height="46" rx="6" fill="#fdeccf" stroke="#c9973f"/>
|
|
<text x="244" y="258" class="h">JobRepository</text>
|
|
<text x="244" y="276" class="c">every Job/Step execution, read/write/skip counts, ExecutionContext</text>
|
|
<line x1="145" y1="90" x2="145" y2="120" stroke="#5b7fc7"/>
|
|
<line x1="370" y1="210" x2="370" y2="240" stroke="#4a9d63"/>
|
|
</svg>
|
|
</figure>
|
|
|
|
The picture above names everything this article uses:
|
|
|
|
- **Job** — a named, ordered list of steps. `productImportJob` in this repository has two:
|
|
import the rows, then report a count.
|
|
- **Step** — one unit of work within a job. A step is either a single **Tasklet** (run
|
|
once, do one thing — `reportStep` here) or **chunk-oriented**: read one item, process it,
|
|
repeat until you have a chunk's worth, then write and commit the whole chunk in one
|
|
transaction.
|
|
- **ItemReader / ItemProcessor / ItemWriter** — the three interfaces chunk-oriented
|
|
processing is built from. This module's reader is a `FlatFileItemReader` over a CSV, the
|
|
processor validates and filters rows, the writer is a `JdbcBatchItemWriter`.
|
|
- **JobRepository** — the thing that makes the first two bullet points of this page
|
|
possible. It persists every job and step execution: status, timestamps, and the read / write /
|
|
filter / skip / commit / rollback counts you saw in the transcripts above. Chapter
|
|
[10](10-resourceless-vs-jdbc.md) is entirely about a trap hiding in how this gets configured on
|
|
Spring Boot 4.1.
|
|
|
|
Hold onto one fact from this page, because it is the one everything else cashes in later: **a
|
|
chunk commits or it doesn't, as a whole.** There is no partial chunk. That single sentence is the
|
|
entire explanation for why restarting a failed job does not reprocess everything, and it is why
|
|
the "poisoned" row in this repository's demo data fails nine other, perfectly good rows along
|
|
with it — see [chapter 7](07-restartability.md).
|
|
|
|
## What this repository demonstrates, and how to run it
|
|
|
|
Four Spring profiles configure the same job three different ways plus one infrastructure
|
|
variant; see the [README](../README.md) for the full table and the exact commands. Everything
|
|
under `docs/output/` was produced by `scripts/capture-scenarios.sh` and `scripts/capture-javap.sh`,
|
|
which `scripts/run-all.sh` runs in sequence.
|
|
|
|
[Next: Anatomy of a job →](02-anatomy-of-a-job.md)
|