Add spring-batch: jobs, steps, chunk processing and restartability on Boot 4.1
Companion code for "Spring Batch on Boot 4.1: Jobs, Steps, Chunk Processing and Restartability". A productImportJob configured three ways by profile against a poisoned CSV row, run as real java -jar processes (not just JUnit) so the restart story is genuine: a chunk fails and rolls back, the process exits, a brand-new JVM against the same file-based H2 database resumes at the exact next unread row (READ_COUNT 20, not 60) and completes. Findings the build pins: - StepBuilder.chunk(int, PlatformTransactionManager) still compiles in Batch 6.0.5 but returns the legacy SimpleStepBuilder; chunk(int) returns the new ChunkOrientedStepBuilder, and only the latter is used here. - Two different ExecutionContext classes now exist in two different packages (infrastructure.item vs core.repository.persistence) with different shapes. - spring-boot-starter-batch alone gives a resourceless JobRepository that forgets every JobInstance the moment the JVM exits; spring-boot-starter- batch-jdbc is what makes the restart demo possible at all, demonstrated by excluding BatchJdbcAutoConfiguration and watching a "restart" collide with the previous run's own data instead of resuming it. - A migration-guide summary claiming CommandLineJobRunner was removed in 6.0 is wrong -- javap against the real jar shows @Deprecated(forRemoval=true), not removed. - RepeatStatus moved from core.repeat to infrastructure.repeat, caught by the compiler rather than by reading docs. 11 documentation chapters, 10 captured transcripts (unit tests, javap output, and real two-JVM scenario runs), all regenerated by scripts/run-all.sh. Fixed after push: three dead docs.spring.io links in the doc chapters (readersAndWriters/* and chunk-oriented-processing/*.html paths moved when Spring Batch 6 reorganized its reference docs; corrected to the current readers-and-writers/*, processor.html and chunk-oriented-processing.html paths, verified 200 via curl before committing). Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_019DXsJ1zpikbA1MQJN6RqFA
This commit is contained in:
Claude
@@ -0,0 +1,86 @@
|
||||
# 1. The problem, and the smallest correct mental model
|
||||
|
||||
[← README](../README.md) | [Next: Anatomy of a job →](02-anatomy-of-a-job.md)
|
||||
|
||||
A batch job processes a lot of records without a person watching each one: importing a CSV of
|
||||
products into a database, closing out a day's transactions, re-indexing a search table. Two
|
||||
things make this harder than writing a `for` loop over the rows:
|
||||
|
||||
- **It has to survive being interrupted.** A process gets killed, a database connection drops,
|
||||
a row three-quarters of the way through is bad. A `for` loop that dies at row 4,700,000 has no
|
||||
idea it already wrote 4,699,999 rows, and re-running it from the top either duplicates work or
|
||||
duplicates data.
|
||||
- **It has to report what happened**, precisely: how many rows were read, how many were written,
|
||||
how many were skipped and why, whether it finished. "The import ran" is not an answer anyone
|
||||
building on top of this can use.
|
||||
|
||||
Spring Batch's job is to own both of those problems so your code only has to describe three
|
||||
things: where the rows come from, what to do to each one, and where they go. Everything else
|
||||
— tracking progress, committing in batches, remembering where a failed run stopped —
|
||||
is the framework's job, provided you tell it enough for it to do that job. Most of this article
|
||||
is about that "provided."
|
||||
|
||||
## The five nouns
|
||||
|
||||
<figure>
|
||||
<svg viewBox="0 0 740 300" role="img" aria-label="A Job contains an ordered list of Steps; each Step is chunk-oriented and reads from a reader, transforms with a processor, and writes with a writer; a JobRepository records everything happening.">
|
||||
<style>
|
||||
.t{font:600 14px sans-serif;fill:#1a1a1a}.h{font:600 12px sans-serif;fill:#1a1a1a}.c{font:11px sans-serif;fill:#4b5563}.m{font:11px monospace;fill:#1a1a1a}
|
||||
</style>
|
||||
<rect x="10" y="10" width="720" height="80" rx="6" fill="#e8eefc" stroke="#5b7fc7"/>
|
||||
<text x="24" y="30" class="h">Job "productImportJob"</text>
|
||||
<text x="24" y="50" class="m">start(importStep).next(reportStep)</text>
|
||||
<rect x="30" y="60" width="180" height="24" rx="4" fill="#fff" stroke="#5b7fc7"/>
|
||||
<text x="45" y="76" class="m">Step: importStep</text>
|
||||
<rect x="230" y="60" width="180" height="24" rx="4" fill="#fff" stroke="#5b7fc7"/>
|
||||
<text x="245" y="76" class="m">Step: reportStep</text>
|
||||
<rect x="60" y="120" width="620" height="90" rx="6" fill="#e7f4ea" stroke="#4a9d63"/>
|
||||
<text x="74" y="140" class="h">importStep is chunk-oriented</text>
|
||||
<rect x="80" y="155" width="150" height="40" rx="4" fill="#fff" stroke="#4a9d63"/>
|
||||
<text x="95" y="179" class="m">ItemReader</text>
|
||||
<rect x="290" y="155" width="150" height="40" rx="4" fill="#fff" stroke="#4a9d63"/>
|
||||
<text x="305" y="179" class="m">ItemProcessor</text>
|
||||
<rect x="500" y="155" width="150" height="40" rx="4" fill="#fff" stroke="#4a9d63"/>
|
||||
<text x="515" y="179" class="m">ItemWriter</text>
|
||||
<line x1="230" y1="175" x2="290" y2="175" stroke="#4a9d63" marker-end="url(#arrow)"/>
|
||||
<line x1="440" y1="175" x2="500" y2="175" stroke="#4a9d63" marker-end="url(#arrow)"/>
|
||||
<defs><marker id="arrow" markerWidth="8" markerHeight="8" refX="6" refY="4" orient="auto"><path d="M0,0 L8,4 L0,8 z" fill="#4a9d63"/></marker></defs>
|
||||
<rect x="230" y="240" width="280" height="46" rx="6" fill="#fdeccf" stroke="#c9973f"/>
|
||||
<text x="244" y="258" class="h">JobRepository</text>
|
||||
<text x="244" y="276" class="c">every Job/Step execution, read/write/skip counts, ExecutionContext</text>
|
||||
<line x1="145" y1="90" x2="145" y2="120" stroke="#5b7fc7"/>
|
||||
<line x1="370" y1="210" x2="370" y2="240" stroke="#4a9d63"/>
|
||||
</svg>
|
||||
</figure>
|
||||
|
||||
The picture above names everything this article uses:
|
||||
|
||||
- **Job** — a named, ordered list of steps. `productImportJob` in this repository has two:
|
||||
import the rows, then report a count.
|
||||
- **Step** — one unit of work within a job. A step is either a single **Tasklet** (run
|
||||
once, do one thing — `reportStep` here) or **chunk-oriented**: read one item, process it,
|
||||
repeat until you have a chunk's worth, then write and commit the whole chunk in one
|
||||
transaction.
|
||||
- **ItemReader / ItemProcessor / ItemWriter** — the three interfaces chunk-oriented
|
||||
processing is built from. This module's reader is a `FlatFileItemReader` over a CSV, the
|
||||
processor validates and filters rows, the writer is a `JdbcBatchItemWriter`.
|
||||
- **JobRepository** — the thing that makes the first two bullet points of this page
|
||||
possible. It persists every job and step execution: status, timestamps, and the read / write /
|
||||
filter / skip / commit / rollback counts you saw in the transcripts above. Chapter
|
||||
[10](10-resourceless-vs-jdbc.md) is entirely about a trap hiding in how this gets configured on
|
||||
Spring Boot 4.1.
|
||||
|
||||
Hold onto one fact from this page, because it is the one everything else cashes in later: **a
|
||||
chunk commits or it doesn't, as a whole.** There is no partial chunk. That single sentence is the
|
||||
entire explanation for why restarting a failed job does not reprocess everything, and it is why
|
||||
the "poisoned" row in this repository's demo data fails nine other, perfectly good rows along
|
||||
with it — see [chapter 7](07-restartability.md).
|
||||
|
||||
## What this repository demonstrates, and how to run it
|
||||
|
||||
Four Spring profiles configure the same job three different ways plus one infrastructure
|
||||
variant; see the [README](../README.md) for the full table and the exact commands. Everything
|
||||
under `docs/output/` was produced by `scripts/capture-scenarios.sh` and `scripts/capture-javap.sh`,
|
||||
which `scripts/run-all.sh` runs in sequence.
|
||||
|
||||
[Next: Anatomy of a job →](02-anatomy-of-a-job.md)
|
||||
@@ -0,0 +1,81 @@
|
||||
# 2. The anatomy of a job, and why the code looks different from older tutorials
|
||||
|
||||
[← Previous](01-the-problem-and-mental-model.md) | [README](../README.md) | [Next: Chunk-oriented processing →](03-chunk-oriented-processing.md)
|
||||
|
||||
`BatchConfig` in this module builds jobs and steps like this:
|
||||
|
||||
```java
|
||||
new JobBuilder("productImportJob", jobRepository)
|
||||
.start(importStep)
|
||||
.next(reportStep)
|
||||
.build();
|
||||
|
||||
new StepBuilder("importStep", jobRepository)
|
||||
.<Product, Product>chunk(10)
|
||||
.transactionManager(transactionManager)
|
||||
.reader(productReader)
|
||||
.processor(productProcessor)
|
||||
.writer(productWriter)
|
||||
.build();
|
||||
```
|
||||
|
||||
If you have seen Spring Batch code before 2023 or so, two things here might look unfamiliar,
|
||||
and it is worth being precise about which is a genuine Spring Boot 4.1 change and which is just
|
||||
old:
|
||||
|
||||
- `JobBuilderFactory` / `StepBuilderFactory` autowired as beans, then called as
|
||||
`jobBuilderFactory.get("name")` — **this was removed in Spring Batch 5, years before this
|
||||
article.** It is not a Boot 4.1 surprise, it is a dead end that tutorials keep copy-pasting.
|
||||
`JobBuilder` and `StepBuilder` are plain classes you construct directly with a `JobRepository`,
|
||||
as above.
|
||||
- `.chunk(10, transactionManager)` — passing the transaction manager as a second argument
|
||||
to `chunk()` — **this compiles in 6.0.5, but it returns a different builder.**
|
||||
`StepBuilder.chunk(int)` returns a `ChunkOrientedStepBuilder`, the model this whole module
|
||||
uses; `StepBuilder.chunk(int, PlatformTransactionManager)` returns the older
|
||||
`SimpleStepBuilder`, kept for the pre-6.0 chunk-processing model. Real `javap` output for both
|
||||
overloads, from this project's own `spring-batch-core-6.0.5.jar`:
|
||||
[`docs/output/03-stepbuilder-chunk-overloads.txt`](output/03-stepbuilder-chunk-overloads.txt).
|
||||
The two builders overlap almost entirely in the methods you would reach for —
|
||||
`reader`/`processor`/`writer`/`faultTolerant`/`skip` exist on both — so the wrong choice
|
||||
usually still compiles and runs, and just quietly uses the legacy step implementation. Call
|
||||
`.chunk(10).transactionManager(tx)` (two calls) if you want `ChunkOrientedStep`.
|
||||
|
||||
<blockquote style="background:#f4f5f7;border:1px solid #e2e5ea;border-left:4px solid #b7bec9;border-radius:6px;padding:16px 20px;"><strong>The fingerprint of picking the wrong overload.</strong> Your step still builds, runs, reads, writes and commits correctly — there is no error. The tell is in what you can't reach: <code>ChunkOrientedStepBuilder</code>-only options like <code>retryPolicy(RetryPolicy)</code> using Spring Framework 7's retry API, or a stack trace mentioning <code>ChunkOrientedStep</code> versus the older <code>TaskletStep</code> wrapping a <code>ChunkOrientedTasklet</code>. If you copied a two-argument <code>chunk(10, txManager)</code> from a pre-6.0 example and it "just worked", this is why nothing complained.</blockquote>
|
||||
|
||||
## Where things moved
|
||||
|
||||
Spring Batch 6.0 (bundled with Spring Boot 4.1.1 as `spring-batch.version` `6.0.5`, confirmed
|
||||
against `spring-boot-dependencies-4.1.1.pom`) reorganized packages fairly aggressively. The ones
|
||||
this module's code actually hits:
|
||||
|
||||
| Old (Spring Batch 5.x) | New (6.0.5) |
|
||||
|---|---|
|
||||
| `org.springframework.batch.core.repeat.RepeatStatus` | `org.springframework.batch.infrastructure.repeat.RepeatStatus` |
|
||||
| `org.springframework.batch.item.*` (readers, writers, `ExecutionContext`) | `org.springframework.batch.infrastructure.item.*` |
|
||||
| `org.springframework.batch.core.JobParameters` (mutable-ish, `Map`-backed) | `org.springframework.batch.core.job.parameters.JobParameters` (immutable record, `Set`-backed) |
|
||||
|
||||
The `RepeatStatus` move is the one this project's own `reportTasklet` hit directly — see
|
||||
[chapter 9](09-corrections.md) for the exact compiler error it produced before the import was
|
||||
fixed. It is a good example of the article's verification rule in practice: rather than trusting
|
||||
a description of where things moved, write the code from the old import, let `javac` say
|
||||
`package ... does not exist`, and fix the import. That is faster and more reliable than
|
||||
reading a migration guide's prose, and chapter 9 has a case where trusting the prose produced a
|
||||
wrong claim.
|
||||
|
||||
## JobRepository and JobOperator
|
||||
|
||||
Two interfaces you inject rather than configure by hand in this module:
|
||||
|
||||
- **`JobRepository`** — every `JobBuilder` and `StepBuilder` above takes one as a
|
||||
constructor argument. In 6.0 it also extends `JobExplorer` (querying past executions), so
|
||||
there is one bean to inject instead of two.
|
||||
- **`JobOperator`** — what `ImportRunner` calls to start the job (see
|
||||
[chapter 5](05-launching-and-jobparameters.md)). It extends `JobLauncher` and adds operational
|
||||
methods: `restart(executionId)`, `stop(executionId)`, `recover(execution)`, `getJobNames()`.
|
||||
`JobLauncher`/`JobExplorer` still exist but are the deprecated half of this pair now.
|
||||
|
||||
Both are auto-configured by `spring-boot-starter-batch-jdbc` on this module's classpath; nothing
|
||||
in `BatchConfig` declares them as beans. [Chapter 10](10-resourceless-vs-jdbc.md) explains what
|
||||
you get instead if that starter is missing.
|
||||
|
||||
[Next: Chunk-oriented processing →](03-chunk-oriented-processing.md)
|
||||
@@ -0,0 +1,78 @@
|
||||
# 3. Chunk-oriented processing: what "chunk(10)" actually does
|
||||
|
||||
[← Previous](02-anatomy-of-a-job.md) | [README](../README.md) | [Next: The item processor as a filter →](04-item-processor-as-filter.md)
|
||||
|
||||
`chunk(10)` means: read up to 10 items (calling the reader once per item), run each through the
|
||||
processor, then hand all the survivors to the writer in **one call**, inside **one transaction**.
|
||||
If the writer succeeds, the transaction commits and the step's counters — read, write,
|
||||
filter, commit — move forward together. If anything in that chunk throws, the whole
|
||||
transaction rolls back: none of those 10 items' writes are kept, even the ones that would have
|
||||
succeeded on their own.
|
||||
|
||||
The happy-path run of this module's `productImportJob` against clean data makes this concrete.
|
||||
60 rows, chunk size 10, two rows fail the processor's validation (filtered, not written):
|
||||
|
||||
```console
|
||||
$ SELECT read_count, filter_count, write_count, commit_count FROM BATCH_STEP_EXECUTION;
|
||||
READ_COUNT | FILTER_COUNT | WRITE_COUNT | COMMIT_COUNT
|
||||
60 | 2 | 58 | 6
|
||||
```
|
||||
|
||||
Full transcript: [`docs/output/05-happy-path.txt`](output/05-happy-path.txt), source:
|
||||
[`ProductValidatingProcessor.java`](../src/main/java/com/ankurm/batch/processing/ProductValidatingProcessor.java).
|
||||
60 rows at chunk size 10 is 6 chunks — `COMMIT_COUNT` confirms all 6 committed, one
|
||||
transaction each.
|
||||
|
||||
<figure>
|
||||
<svg viewBox="0 0 740 220" role="img" aria-label="60 rows split into six chunks of 10; each chunk is read, processed, and written as one transaction that either commits fully or rolls back fully.">
|
||||
<style>.h{font:600 12px sans-serif;fill:#1a1a1a}.c{font:11px sans-serif;fill:#4b5563}.m{font:11px monospace;fill:#1a1a1a}</style>
|
||||
<text x="20" y="24" class="h">60 rows → six chunks of 10, each its own transaction</text>
|
||||
<g>
|
||||
<!-- 6 chunk boxes -->
|
||||
<rect x="20" y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
|
||||
<rect x="140" y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
|
||||
<rect x="260" y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
|
||||
<rect x="380" y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
|
||||
<rect x="500" y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
|
||||
<rect x="620" y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
|
||||
<text x="45" y="80" class="m">1-10</text>
|
||||
<text x="165" y="80" class="m">11-20</text>
|
||||
<text x="285" y="80" class="m">21-30</text>
|
||||
<text x="405" y="80" class="m">31-40</text>
|
||||
<text x="525" y="80" class="m">41-50</text>
|
||||
<text x="645" y="80" class="m">51-60</text>
|
||||
</g>
|
||||
<text x="20" y="130" class="c">Each box: read 10 → process 10 (0-1 filtered) → write survivors → COMMIT.</text>
|
||||
<text x="20" y="150" class="c">Row 12 (chunk 2) and row 33 (chunk 4) are filtered by the processor -- see chapter 4 -- so those</text>
|
||||
<text x="20" y="170" class="c">two chunks commit 9 rows instead of 10. 58 rows land in PRODUCT; every chunk still commits.</text>
|
||||
</svg>
|
||||
</figure>
|
||||
|
||||
## Why 10, and what changes if you pick a different size
|
||||
|
||||
Chunk size is a trade-off with no universally right answer:
|
||||
|
||||
- **Smaller chunks** commit more often, so a failure loses less work and transactions are
|
||||
shorter (less lock contention, smaller rollback). They also mean more round trips to the
|
||||
database — more commit overhead per row.
|
||||
- **Larger chunks** amortize that overhead across more rows, but a failure anywhere in the chunk
|
||||
costs you the whole chunk's work, and the transaction held open is bigger and longer.
|
||||
|
||||
10 is small enough that chapter 7's poisoned row costs you 12 rows of rework at most (one
|
||||
chunk), not thousands, and small enough to demonstrate multiple commits from 60 rows without a
|
||||
huge fixture file. Production jobs processing millions of rows commonly use chunk sizes in the
|
||||
hundreds or low thousands; the right number depends on row size, write cost, and how expensive a
|
||||
partial redo is for your specific job — there is no formula that replaces measuring it
|
||||
against your own writer.
|
||||
|
||||
## Going deeper
|
||||
|
||||
- The chunk-processing reference: [Spring Batch — Chunk-oriented processing](https://docs.spring.io/spring-batch/reference/step/chunk-oriented-processing.html) (`rel="nofollow"`).
|
||||
- `ChunkOrientedStep` itself, if you want to see the commit/rollback logic this chapter
|
||||
describes: `org.springframework.batch.core.step.item.ChunkOrientedStep` in
|
||||
`spring-batch-core-6.0.5.jar` (the stack trace in
|
||||
[`docs/output/07-restart-run1-fails.txt`](output/07-restart-run1-fails.txt) names it directly).
|
||||
- What happens when the processor itself decides to drop a row rather than the writer failing:
|
||||
[chapter 4](04-item-processor-as-filter.md).
|
||||
|
||||
[Next: The item processor as a filter →](04-item-processor-as-filter.md)
|
||||
@@ -0,0 +1,62 @@
|
||||
# 4. The item processor as a filter
|
||||
|
||||
[← Previous](03-chunk-oriented-processing.md) | [README](../README.md) | [Next: Launching and JobParameters →](05-launching-and-jobparameters.md)
|
||||
|
||||
`ItemProcessor<I, O>` has one method: `O process(I item)`. Returning a transformed item is the
|
||||
obvious case; returning `null` is the one people miss, and it means something specific:
|
||||
**this item is filtered.** It is counted separately (`FILTER_COUNT`) from both `READ_COUNT` and
|
||||
`WRITE_COUNT`, and it never reaches the writer, so it cannot throw the exception a bad write
|
||||
would.
|
||||
|
||||
[`ProductValidatingProcessor`](../src/main/java/com/ankurm/batch/processing/ProductValidatingProcessor.java)
|
||||
uses this for two checks that can be made from one row alone, with no database involved:
|
||||
|
||||
```java
|
||||
if (!SKU_PATTERN.matcher(item.sku()).matches()) {
|
||||
return null; // filtered: does not match ^[A-Z]{3}-\d{4}$
|
||||
}
|
||||
if (item.priceCents() <= 0) {
|
||||
return null; // filtered: priceCents must be positive
|
||||
}
|
||||
```
|
||||
|
||||
Real output, from a plain unit test with no Spring context involved
|
||||
([`ProductValidatingProcessorTest.java`](../src/test/java/com/ankurm/batch/ProductValidatingProcessorTest.java),
|
||||
transcript [`docs/output/01-processor-filter.txt`](output/01-processor-filter.txt)):
|
||||
|
||||
```console
|
||||
input : Product[sku=abc-0012, name=Widget 12, priceCents=1012] <- lowercase sku, fails ^[A-Z]{3}-\d{4}$
|
||||
result : null
|
||||
|
||||
input : Product[sku=ABC-0033, name=Widget 33, priceCents=0] <- priceCents is zero
|
||||
result : null
|
||||
```
|
||||
|
||||
## Filter vs. skip vs. fail: pick by what the check needs
|
||||
|
||||
This module deliberately does not filter the poisoned duplicate SKU in the processor, even
|
||||
though duplication sounds like a validation problem. The reason is what information the check
|
||||
needs:
|
||||
|
||||
| Check | What it needs | Where it belongs |
|
||||
|---|---|---|
|
||||
| SKU matches a format | Just this row | Processor, filter (`null`) |
|
||||
| Price is positive | Just this row | Processor, filter (`null`) |
|
||||
| SKU is unique | Every other row already written | The database, via a constraint |
|
||||
|
||||
A processor can only see one item at a time (by design — that is what lets Spring Batch
|
||||
process items independently and in chunks). It cannot know a SKU is a duplicate without querying
|
||||
everything written so far, which defeats the point of a stateless per-item check and does not
|
||||
scale. The database already tracks this via the `UNIQUE` constraint on `PRODUCT.sku`
|
||||
([`schema.sql`](../src/main/resources/schema.sql)), so that is where the duplicate check
|
||||
actually happens — as a write failure, not a filtered row. What a write failure does to the
|
||||
rest of the chunk, and how the job recovers from it, is the rest of this article: [chapter 7](07-restartability.md) and [chapter 8](08-skip-vs-restart.md).
|
||||
|
||||
## Going deeper
|
||||
|
||||
- `ItemProcessor` returning `null`: [Spring Batch reference — item processing](https://docs.spring.io/spring-batch/reference/processor.html) (`rel="nofollow"`).
|
||||
- Chaining processors with `CompositeItemProcessor` when validation and transformation both need
|
||||
their own class — not used in this module (one processor does both jobs here), but worth
|
||||
knowing before a single processor method grows past a screenful.
|
||||
|
||||
[Next: Launching and JobParameters →](05-launching-and-jobparameters.md)
|
||||
@@ -0,0 +1,65 @@
|
||||
# 5. Launching a job, and why restart is not a separate API
|
||||
|
||||
[← Previous](04-item-processor-as-filter.md) | [README](../README.md) | [Next: The JDBC writer and Java records →](06-jdbc-writer-and-records.md)
|
||||
|
||||
[`ImportRunner`](../src/main/java/com/ankurm/batch/runner/ImportRunner.java) launches the job
|
||||
with:
|
||||
|
||||
```java
|
||||
var params = new JobParametersBuilder()
|
||||
.addString("batch.run", "demo") // identifying
|
||||
.toJobParameters();
|
||||
try {
|
||||
var execution = jobOperator.start(job, params);
|
||||
} catch (JobInstanceAlreadyCompleteException e) {
|
||||
// already succeeded; nothing to do
|
||||
}
|
||||
```
|
||||
|
||||
`JobParametersBuilder` and `addString(...)` look exactly like Spring Batch 4 and 5. What changed
|
||||
underneath is the type it builds: `JobParameters` is now an immutable record holding a
|
||||
`Set<JobParameter<?>>`, and each `JobParameter<T>` is itself a record carrying its own name,
|
||||
value, type, and an `identifying` flag (default `true` for the `addX` methods used here). None of
|
||||
that shows up in this code — it matters if you ever construct a `JobParameters` by hand
|
||||
instead of through the builder, or serialize one.
|
||||
|
||||
## "Identifying" is the whole restart mechanism
|
||||
|
||||
A **JobInstance** is identified by a job name plus the set of parameters marked `identifying`.
|
||||
Calling `jobOperator.start(job, params)` with parameters that match an existing JobInstance does
|
||||
not create a second, independent run:
|
||||
|
||||
- If that instance's last execution is not `COMPLETED` (it `FAILED`, or never finished), Spring
|
||||
Batch creates a new `JobExecution` **against the same instance** and each step resumes from
|
||||
its own last-committed position. This is a restart, and it happens through the exact same
|
||||
`start()` call as a first attempt — see [chapter 7](07-restartability.md) for what
|
||||
"resumes from" means concretely.
|
||||
- If that instance's last execution `COMPLETED`, `start()` throws
|
||||
`JobInstanceAlreadyCompleteException` (unless the step allows re-running —
|
||||
`allowStartIfComplete`, used on `reportStep`; see [chapter 8](08-skip-vs-restart.md)).
|
||||
|
||||
There is a separate `JobOperator.restart(long executionId)` method for restarting by execution
|
||||
ID explicitly, useful for an operational tool that lists failed executions and lets someone pick
|
||||
one. This module does not need it: every run uses the same identifying parameter
|
||||
(`batch.run=demo`), so plain `start()` already does the right thing whether this is attempt one
|
||||
or attempt two. If you want a fresh JobInstance on every run instead — the common pattern
|
||||
for "run this daily" jobs — add a parameter that changes each time, typically a timestamp
|
||||
or an incrementer, and mark it identifying (the default).
|
||||
|
||||
## `spring.batch.job.enabled`
|
||||
|
||||
Spring Boot auto-configures a `JobLauncherApplicationRunner` that launches every `Job` bean it
|
||||
finds using empty parameters, on every application startup. This module turns that off
|
||||
(`spring.batch.job.enabled: false` in [`application.yml`](../src/main/resources/application.yml))
|
||||
because `ImportRunner` needs to control the parameters (the identifying `batch.run` value) and
|
||||
which profile's `Job` bean is active. Leaving both runners on would launch the job twice per
|
||||
startup with two different parameter sets. If you only ever need "run the one job on startup
|
||||
with no arguments," the auto-configured runner is simpler and this whole class is unnecessary.
|
||||
|
||||
## Going deeper
|
||||
|
||||
- `JobParameters` and identifying parameters: [Spring Batch reference — running a job](https://docs.spring.io/spring-batch/reference/job/running.html) (`rel="nofollow"`).
|
||||
- `spring.batch.job.name` for selecting which job the auto-configured runner launches, when you
|
||||
have more than one `Job` bean and want to keep using it.
|
||||
|
||||
[Next: The JDBC writer and Java records →](06-jdbc-writer-and-records.md)
|
||||
@@ -0,0 +1,59 @@
|
||||
# 6. The JDBC writer, and why it is not `beanMapped()`
|
||||
|
||||
[← Previous](05-launching-and-jobparameters.md) | [README](../README.md) | [Next: Restartability →](07-restartability.md)
|
||||
|
||||
`JdbcBatchItemWriterBuilder` offers two ways to map an item's fields to SQL parameters:
|
||||
`columnMapped()` (positional, via `Map`/`SqlParameterSource` keyed by column name) and
|
||||
`beanMapped()` (reflective, via `BeanPropertySqlParameterSource`, matching `:paramName` markers
|
||||
in the SQL to JavaBean getters). Most Spring Batch examples reach for `beanMapped()` because it
|
||||
needs the least code.
|
||||
|
||||
`Product` in this module is a Java record:
|
||||
|
||||
```java
|
||||
public record Product(String sku, String name, long priceCents) {}
|
||||
```
|
||||
|
||||
Its accessors are `sku()`, `name()`, `priceCents()` — no `get` prefix. Standard JavaBean
|
||||
introspection, which `BeanPropertySqlParameterSource` uses, looks for `getSku()`, `getName()`,
|
||||
`getPriceCents()`. Those do not exist on a record, so `beanMapped()` against a plain record either
|
||||
finds nothing to bind (leaving every parameter `NULL`) or fails outright, depending on the exact
|
||||
introspector version in play — not a mistake you want to discover from a table full of NULLs
|
||||
in production.
|
||||
|
||||
[`BatchConfig.productWriter`](../src/main/java/com/ankurm/batch/config/BatchConfig.java) sidesteps
|
||||
the question entirely with `itemPreparedStatementSetter`, setting each column explicitly:
|
||||
|
||||
```java
|
||||
new JdbcBatchItemWriterBuilder<Product>()
|
||||
.dataSource(jdbcTemplate.getDataSource())
|
||||
.sql("INSERT INTO PRODUCT (sku, name, price_cents) VALUES (?, ?, ?)")
|
||||
.itemPreparedStatementSetter((item, ps) -> {
|
||||
ps.setString(1, item.sku());
|
||||
ps.setString(2, item.name());
|
||||
ps.setLong(3, item.priceCents());
|
||||
})
|
||||
.assertUpdates(true)
|
||||
.build();
|
||||
```
|
||||
|
||||
More typing, zero ambiguity about what gets bound where, and it works identically whether the
|
||||
item type is a record, a plain class, or something with no JavaBean getters at all.
|
||||
|
||||
<blockquote style="background:#f4f5f7;border:1px solid #e2e5ea;border-left:4px solid #b7bec9;border-radius:6px;padding:16px 20px;"><strong>If you want <code>beanMapped()</code> with records anyway.</strong> Newer versions of Spring's <code>BeanWrapperImpl</code> (the machinery behind <code>BeanPropertySqlParameterSource</code>) have gained some record support in recent Spring Framework releases, but the safe rule is to check it against the exact Spring Framework version you are on rather than assume — the failure mode when it does not work is silent <code>NULL</code>s, not an exception, which is the worst kind of wrong.</blockquote>
|
||||
|
||||
`assertUpdates(true)` is worth keeping on deliberately: it makes the writer throw if a batch
|
||||
statement reports zero rows updated for any item, instead of silently accepting a no-op write.
|
||||
Combined with `itemPreparedStatementSetter`, a typo in a column name fails loudly at the first
|
||||
write attempt rather than producing a table that looks plausible but is missing a column's worth
|
||||
of data.
|
||||
|
||||
## Going deeper
|
||||
|
||||
- `JdbcBatchItemWriter` and its two mapping styles: [Spring Batch reference — item writers](https://docs.spring.io/spring-batch/reference/readers-and-writers/item-writer.html) (`rel="nofollow"`).
|
||||
- Java records and JavaBean introspection generally: this is not a Spring Batch quirk, it is how
|
||||
`java.beans.Introspector` has always worked, and it bites anywhere a library assumes
|
||||
`getX()`/`setX()` — Jackson, JPA, validation frameworks — each with its own answer
|
||||
for how much (if any) record support it has added.
|
||||
|
||||
[Next: Restartability →](07-restartability.md)
|
||||
@@ -0,0 +1,112 @@
|
||||
# 7. Restartability: what actually resumes, and from where
|
||||
|
||||
[← Previous](06-jdbc-writer-and-records.md) | [README](../README.md) | [Next: Skip vs. restart →](08-skip-vs-restart.md)
|
||||
|
||||
This chapter is the "run 1 fails, run 2 in a brand-new JVM finishes the job" demonstration
|
||||
[the article](https://ankurm.com/) leads with. Two real, separate `java -jar` invocations,
|
||||
against the same file-based H2 database, produced everything below — see
|
||||
[`scripts/capture-scenarios.sh`](../scripts/capture-scenarios.sh) for the exact commands.
|
||||
|
||||
## Run 1: the poisoned duplicate fails a whole chunk
|
||||
|
||||
`products-poison.csv` is identical to the clean fixture except row 47, where the SKU is changed
|
||||
to `ABC-0005` — a duplicate of row 5. Row 47 falls in chunk 5 (rows 41-50 at chunk size
|
||||
10). The `broken` profile's step has no fault tolerance configured, so the `UNIQUE` constraint
|
||||
violation on that insert fails the batch statement, which fails the chunk, which fails the step,
|
||||
which fails the job:
|
||||
|
||||
```console
|
||||
$ SELECT status, read_count, filter_count, write_count, commit_count, rollback_count FROM BATCH_STEP_EXECUTION;
|
||||
STATUS | READ_COUNT | FILTER_COUNT | WRITE_COUNT | COMMIT_COUNT | ROLLBACK_COUNT
|
||||
FAILED | 50 | 2 | 38 | 4 | 1
|
||||
```
|
||||
|
||||
Full transcript: [`docs/output/07-restart-run1-fails.txt`](output/07-restart-run1-fails.txt).
|
||||
Four chunks (rows 1-40, minus the 2 filtered by the processor — see
|
||||
[chapter 4](04-item-processor-as-filter.md) — = 38 written) committed successfully before
|
||||
chunk 5 rolled back entirely. The reader had read through row 50 by the time the writer failed
|
||||
(`READ_COUNT` 50), but none of chunk 5's rows are in the `PRODUCT` table — the transaction
|
||||
that would have written them rolled back along with everything else in that chunk.
|
||||
|
||||
## The fix, and the restart
|
||||
|
||||
Someone fixes the source data — here, changing row 47's SKU to something that is not a
|
||||
duplicate, in place, same line, same file:
|
||||
|
||||
```console
|
||||
sed -i '48s/ABC-0005/ABC-0999/' scenario-data/restart-demo/input.csv
|
||||
```
|
||||
|
||||
Then the exact same command that produced run 1 runs again, in a completely new JVM process,
|
||||
using the exact same identifying job parameter (`batch.run=demo`, see
|
||||
[chapter 5](05-launching-and-jobparameters.md)):
|
||||
|
||||
```console
|
||||
$ SELECT step_execution_id, job_execution_id, status, read_count, write_count, commit_count, rollback_count FROM BATCH_STEP_EXECUTION ORDER BY step_execution_id;
|
||||
STEP_EXECUTION_ID | JOB_EXECUTION_ID | STATUS | READ_COUNT | WRITE_COUNT | COMMIT_COUNT | ROLLBACK_COUNT
|
||||
1 | 1 | FAILED | 50 | 38 | 4 | 1
|
||||
2 | 2 | COMPLETED | 20 | 20 | 2 | 0
|
||||
|
||||
$ SELECT COUNT(*) FROM PRODUCT;
|
||||
COUNT(*)
|
||||
58
|
||||
```
|
||||
|
||||
Full transcript: [`docs/output/08-restart-run2-resumes.txt`](output/08-restart-run2-resumes.txt).
|
||||
Read that second row carefully: **`READ_COUNT` is 20, not 60.** The new execution did not start
|
||||
the CSV from row 1. It read only rows 41 through 60 — the two chunks that had not committed
|
||||
yet — wrote all 20 of them (the earlier two filtered rows were both in the first 40, so
|
||||
nothing left to filter here), and the job status flips to `COMPLETED`. `PRODUCT` ends up with
|
||||
58 rows: the 38 that survived run 1, plus these 20.
|
||||
|
||||
<figure>
|
||||
<svg viewBox="0 0 740 260" role="img" aria-label="Run 1 commits chunks 1 through 4 (rows 1-40) then fails on chunk 5; run 2, a fresh JVM against the same database, resumes at row 41 and reads only 20 more rows to finish.">
|
||||
<style>.h{font:600 12px sans-serif;fill:#1a1a1a}.c{font:11px sans-serif;fill:#4b5563}.m{font:11px monospace;fill:#1a1a1a}</style>
|
||||
<text x="20" y="24" class="h">Run 1 (JobExecution 1)</text>
|
||||
<rect x="20" y="40" width="88" height="40" rx="4" fill="#e7f4ea" stroke="#4a9d63"/><text x="34" y="64" class="m">1-10</text>
|
||||
<rect x="116" y="40" width="88" height="40" rx="4" fill="#e7f4ea" stroke="#4a9d63"/><text x="130" y="64" class="m">11-20</text>
|
||||
<rect x="212" y="40" width="88" height="40" rx="4" fill="#e7f4ea" stroke="#4a9d63"/><text x="226" y="64" class="m">21-30</text>
|
||||
<rect x="308" y="40" width="88" height="40" rx="4" fill="#e7f4ea" stroke="#4a9d63"/><text x="322" y="64" class="m">31-40</text>
|
||||
<rect x="404" y="40" width="88" height="40" rx="4" fill="#f7d9d3" stroke="#c56a54"/><text x="415" y="64" class="m">41-50 FAIL</text>
|
||||
<rect x="500" y="40" width="88" height="40" rx="4" fill="#f4f5f7" stroke="#b7bec9" stroke-dasharray="3 3"/><text x="512" y="64" class="c">never read</text>
|
||||
<text x="20" y="105" class="c">committed: rows 1-40 (38 written after 2 filtered) -- job FAILS at chunk 5, exit status FAILED</text>
|
||||
|
||||
<text x="20" y="150" class="h">Run 2 (JobExecution 2) -- fresh JVM, same database, corrected row 47</text>
|
||||
<rect x="20" y="166" width="380" height="40" rx="4" fill="#f4f5f7" stroke="#b7bec9" stroke-dasharray="3 3"/><text x="150" y="190" class="c">rows 1-40: already committed, NOT re-read</text>
|
||||
<rect x="404" y="166" width="88" height="40" rx="4" fill="#e7f4ea" stroke="#4a9d63"/><text x="415" y="190" class="m">41-50 OK</text>
|
||||
<rect x="500" y="166" width="88" height="40" rx="4" fill="#e7f4ea" stroke="#4a9d63"/><text x="511" y="190" class="m">51-60 OK</text>
|
||||
<text x="20" y="230" class="c">READ_COUNT for this execution = 20, not 60. COMPLETED. PRODUCT now has 58 rows total.</text>
|
||||
</svg>
|
||||
</figure>
|
||||
|
||||
## Where the resume position actually lives
|
||||
|
||||
The reader's position is not something Spring Batch infers after the fact — it is saved,
|
||||
by the reader itself, into the step's `ExecutionContext` at every chunk commit, in the **same
|
||||
transaction** as the chunk's writes. `FlatFileItemReader` implements `ItemStream`
|
||||
(`org.springframework.batch.infrastructure.item.ItemStream`, see
|
||||
[`docs/output/04-two-executioncontext-classes.txt`](output/04-two-executioncontext-classes.txt)
|
||||
for its shape) and calls `update(ExecutionContext)` on every commit, storing the current line
|
||||
count. Because that update happens inside the chunk's transaction:
|
||||
|
||||
- If the chunk commits, the new line count is persisted right alongside the rows it wrote.
|
||||
- If the chunk rolls back (chunk 5, run 1), the line-count update rolls back too — the
|
||||
persisted position stays at the end of chunk 4, not partway through chunk 5.
|
||||
|
||||
That is the entire mechanism. There is no separate "resume point" concept to configure: it falls
|
||||
out of transactional chunk commits plus a reader that participates in `ItemStream`. Writing a
|
||||
custom reader that does not save its position to the `ExecutionContext` — a plain
|
||||
`ItemReader` with no `ItemStream` — means restarts always start that reader from scratch,
|
||||
which is sometimes exactly what you want (an idempotent reader) and sometimes a bug you will not
|
||||
notice until the first real failure in production.
|
||||
|
||||
## Going deeper
|
||||
|
||||
- `ItemStream` and the two-argument `open`/`update` contract:
|
||||
[Spring Batch reference — readers and ItemStream](https://docs.spring.io/spring-batch/reference/readers-and-writers/item-reader.html) (`rel="nofollow"`).
|
||||
- Configuring restart limits per step (`startLimit`, `allowStartIfComplete`):
|
||||
[chapter 8](08-skip-vs-restart.md), and the [Spring Batch reference on restart configuration](https://docs.spring.io/spring-batch/reference/step/chunk-oriented-processing/restart.html) (`rel="nofollow"`).
|
||||
- What happens if the job repository itself does not persist across the JVM restart shown above:
|
||||
[chapter 10](10-resourceless-vs-jdbc.md) — this entire chapter depends on it being real.
|
||||
|
||||
[Next: Skip vs. restart →](08-skip-vs-restart.md)
|
||||
@@ -0,0 +1,92 @@
|
||||
# 8. Skip vs. restart, and the two knobs that shape a restart
|
||||
|
||||
[← Previous](07-restartability.md) | [README](../README.md) | [Next: Corrections found while writing this →](09-corrections.md)
|
||||
|
||||
Failing the job and restarting after a fix (chapter 7) is one answer to a bad row. It is the
|
||||
right one when the row is genuinely wrong and someone needs to decide what to do about it. It is
|
||||
the wrong one when the bad row is expected background noise — a handful of duplicates in a
|
||||
million-row feed — and stopping the whole job for each one is not realistic.
|
||||
|
||||
The `skip` profile configures the same step with `faultTolerant().skip(...)` instead:
|
||||
|
||||
```java
|
||||
new StepBuilder("importStep", jobRepository)
|
||||
.<Product, Product>chunk(10)
|
||||
.transactionManager(transactionManager)
|
||||
.reader(productReader)
|
||||
.processor(productProcessor)
|
||||
.writer(productWriter)
|
||||
.faultTolerant()
|
||||
.skip(DataIntegrityViolationException.class)
|
||||
.skipLimit(3)
|
||||
.listener(skipListener())
|
||||
.build();
|
||||
```
|
||||
|
||||
Run against the same poisoned file used in chapter 7, in one pass, no restart needed:
|
||||
|
||||
```console
|
||||
$ SELECT status, read_count, filter_count, write_count, write_skip_count, commit_count, rollback_count FROM BATCH_STEP_EXECUTION;
|
||||
COMPLETED | 60 | 2 | 57 | 1 | 6 | 2
|
||||
|
||||
$ SELECT COUNT(*) FROM PRODUCT;
|
||||
COUNT(*)
|
||||
57
|
||||
```
|
||||
|
||||
Full transcript: [`docs/output/10-skip-instead-of-fail.txt`](output/10-skip-instead-of-fail.txt).
|
||||
The job reads all 60 rows in one execution, filters the same 2 as always, skips exactly the
|
||||
duplicate (`WRITE_SKIP_COUNT` 1), and completes with 57 rows written. Note `ROLLBACK_COUNT` is 2,
|
||||
not 0: when a fault-tolerant chunk's batch write fails, Spring Batch does not give up on the
|
||||
chunk — it rolls back once, then re-processes that chunk's items **one at a time** to find
|
||||
out which single item is the actual problem, so it can skip only that one and keep the rest. That
|
||||
retry-by-scanning is one extra rollback for the chunk containing the bad row; you can see it in
|
||||
the count.
|
||||
|
||||
## `DataIntegrityViolationException.class` catches `DuplicateKeyException` because of the hierarchy
|
||||
|
||||
The skip policy is registered against `DataIntegrityViolationException`, but the exception H2
|
||||
actually throws is `DuplicateKeyException`. This works because
|
||||
`DuplicateKeyException extends DataIntegrityViolationException` — Spring's JDBC exception
|
||||
translation maps H2's `JdbcBatchUpdateException` (a unique-constraint violation) to the more
|
||||
specific subclass, and a skip policy registered against the parent still matches it. Confirmed
|
||||
directly, not assumed:
|
||||
|
||||
```console
|
||||
thrown type : org.springframework.dao.DuplicateKeyException
|
||||
is a DataIntegrityViolationException? true
|
||||
```
|
||||
|
||||
(from [`docs/output/02-exception-hierarchy.txt`](output/02-exception-hierarchy.txt), produced by
|
||||
[`ProductValidatingProcessorTest.duplicateKeyExceptionIsADataIntegrityViolationException`](../src/test/java/com/ankurm/batch/ProductValidatingProcessorTest.java)).
|
||||
Registering skip policies against a broad parent type like `DataIntegrityViolationException`
|
||||
means you do not have to enumerate every specific subtype Spring's translation layer might
|
||||
produce — but it also means you are choosing to skip *any* data-integrity problem, not just
|
||||
duplicates. `skipLimit(3)` is the safety valve: past 3 skips in one execution, the step gives up
|
||||
and fails anyway, on the theory that "one bad row" and "the whole feed is corrupt" should not be
|
||||
handled identically.
|
||||
|
||||
## The two restart-shaping options on a step
|
||||
|
||||
`reportStep` uses one of these; the football-job example in the Spring Batch reference documents
|
||||
both well, and this module's `reportStep` is a direct, smaller version of that pattern:
|
||||
|
||||
- **`allowStartIfComplete(true)`** — by default, a step that already finished `COMPLETED`
|
||||
is skipped entirely on restart (the whole point of chapter 7 is not re-doing committed work).
|
||||
`reportStep` overrides that, because a status report should reflect the current state of the
|
||||
`PRODUCT` table on every run, including a restart, not just the first successful attempt.
|
||||
- **`startLimit(n)`** — not used in this module, but worth knowing: caps how many times a
|
||||
step may be *attempted* (not completed) before Spring Batch throws
|
||||
`StartLimitExceededException` instead of trying again. Useful for a step that does something
|
||||
non-idempotent enough that repeated automatic retries would make things worse — call out
|
||||
external systems, send notifications — where the right response to repeated failure is a
|
||||
human looking at it, not an unbounded retry loop.
|
||||
|
||||
## Going deeper
|
||||
|
||||
- Skip and retry configuration in full, including `noSkip()` exclusions and combining skip with
|
||||
retry: [Spring Batch reference — configuring skip logic](https://docs.spring.io/spring-batch/reference/step/chunk-oriented-processing/configuring-skip.html) (`rel="nofollow"`).
|
||||
- `startLimit` and `allowStartIfComplete` with the football-job example this chapter borrows the
|
||||
shape of: [Spring Batch reference — configuring a step for restart](https://docs.spring.io/spring-batch/reference/step/chunk-oriented-processing/restart.html) (`rel="nofollow"`).
|
||||
|
||||
[Next: Corrections found while writing this →](09-corrections.md)
|
||||
@@ -0,0 +1,57 @@
|
||||
# 9. Corrections found while writing this
|
||||
|
||||
[← Previous](08-skip-vs-restart.md) | [README](../README.md) | [Next: Resourceless vs. JDBC-backed →](10-resourceless-vs-jdbc.md)
|
||||
|
||||
Per the verification discipline this repository follows, here is what was wrong in an earlier
|
||||
draft, and how it was caught — recorded honestly rather than silently fixed, because the
|
||||
mistake is often as informative as the correct answer.
|
||||
|
||||
## "CommandLineJobRunner was removed in 6.0" — wrong, it is deprecated
|
||||
|
||||
A community-written Spring Batch 6.0 migration guide describes `CommandLineJobRunner` as removed
|
||||
outright, in a list alongside classes that genuinely were deleted
|
||||
(`ChunkListenerSupport`, `JobExecutionListenerSupport`, and others). An early draft of this
|
||||
article repeated that claim. Checking the actual `spring-batch-core-6.0.5.jar` shows it is wrong:
|
||||
|
||||
```console
|
||||
$ javap -verbose -cp spring-batch-core-6.0.5.jar org.springframework.batch.core.launch.support.CommandLineJobRunner | grep -A3 "^public class\|Deprecated"
|
||||
public class org.springframework.batch.core.launch.support.CommandLineJobRunner
|
||||
...
|
||||
Deprecated: true
|
||||
RuntimeVisibleAnnotations:
|
||||
0: #505(#506=s#507,#508=Z#509)
|
||||
java.lang.Deprecated(
|
||||
since="6.0"
|
||||
forRemoval=true
|
||||
)
|
||||
```
|
||||
|
||||
Full transcript: [`docs/output/09-commandlinejobrunner-deprecated-not-removed.txt`](output/09-commandlinejobrunner-deprecated-not-removed.txt).
|
||||
The class is present, loadable, and functional in 6.0.5 — it carries
|
||||
`@Deprecated(since = "6.0", forRemoval = true)`, which means "stop using this, it will be
|
||||
deleted in a future release," not "this is already gone." `CommandLineJobOperator` is the
|
||||
replacement, and it does exist alongside it in the same package.
|
||||
|
||||
This is the article's own self-correction pass working as intended: a claim sourced from a
|
||||
third-party summary of a migration guide, not from a primary artifact, was treated as unverified
|
||||
until checked. The general rule this project follows (see the parent repository's process notes)
|
||||
is that a migration guide's own prose is not itself a primary source for "removed vs.
|
||||
deprecated" — the jar is.
|
||||
|
||||
## `RepeatStatus` package move, caught by the compiler rather than by reading docs
|
||||
|
||||
Writing [`BatchConfig.reportTasklet`](../src/main/java/com/ankurm/batch/config/BatchConfig.java),
|
||||
the first draft imported `org.springframework.batch.core.repeat.RepeatStatus` — the
|
||||
Spring Batch 5.x location, from memory and from older examples. `mvn -B -o compile` reported:
|
||||
|
||||
```
|
||||
[ERROR] .../BatchConfig.java:[93,57] package org.springframework.batch.core.repeat does not exist
|
||||
```
|
||||
|
||||
The fix was `org.springframework.batch.infrastructure.repeat.RepeatStatus`, matching the general
|
||||
package reorganization covered in [chapter 2](02-anatomy-of-a-job.md). No migration guide was
|
||||
consulted for this one; the compiler error was sufficient, which is the point — letting
|
||||
`javac` tell you where an API moved is faster and more reliable than trying to remember or look
|
||||
up a package reorganization table.
|
||||
|
||||
[Next: Resourceless vs. JDBC-backed →](10-resourceless-vs-jdbc.md)
|
||||
@@ -0,0 +1,71 @@
|
||||
# 10. Resourceless vs. JDBC-backed: the dependency that makes chapter 7 possible
|
||||
|
||||
[← Previous](09-corrections.md) | [README](../README.md) | [Next: Production checklist →](11-production-checklist.md)
|
||||
|
||||
Everything in [chapter 7](07-restartability.md) depends on one dependency choice that is easy to
|
||||
get wrong on Spring Boot 4.1, because getting it wrong does not produce an error — the
|
||||
application starts, the job runs, nothing complains.
|
||||
|
||||
Spring Boot 4.1 split Batch autoconfiguration into separate modules rather than one monolith:
|
||||
|
||||
| Starter | Autoconfiguration class | What it provides |
|
||||
|---|---|---|
|
||||
| `spring-boot-starter-batch` | `BatchAutoConfiguration` | Batch infrastructure basics; a **resourceless** (in-memory, non-persistent) `JobRepository` if nothing more specific is configured |
|
||||
| `spring-boot-starter-batch-jdbc` | `BatchJdbcAutoConfiguration` | A real JDBC-backed `JobRepository`, plus `spring.batch.jdbc.initialize-schema` to create the `BATCH_*` tables |
|
||||
|
||||
`spring-boot-starter-batch-jdbc` (this module's actual dependency, in
|
||||
[`pom.xml`](../pom.xml)) pulls in `spring-boot-starter-batch` transitively, so **both**
|
||||
autoconfiguration classes are normally on the classpath together, and `BatchJdbcAutoConfiguration`
|
||||
wins by providing the more specific `JobRepository` bean. Nothing about compiling or starting the
|
||||
application tells you which one actually won.
|
||||
|
||||
## Proving it, by turning the JDBC half off
|
||||
|
||||
`--spring.autoconfigure.exclude=...BatchJdbcAutoConfiguration` removes the JDBC-backed
|
||||
`JobRepository` from the picture, leaving `BatchAutoConfiguration`'s resourceless one as the only
|
||||
candidate. Same job, same identifying job parameters, same `PRODUCT` table, run twice, in two
|
||||
separate JVMs:
|
||||
|
||||
```console
|
||||
-- run 1 --
|
||||
REPORT: 58 products now in the PRODUCT table
|
||||
JOB FINISHED: status=COMPLETED exitCode=COMPLETED
|
||||
|
||||
-- run 2: a second, completely fresh JVM, same jar, same job parameters, same PRODUCT table --
|
||||
org.springframework.dao.DuplicateKeyException: ... Unique index or primary key violation ...
|
||||
VALUES ( /* 1 */ 'ABC-0001' )
|
||||
JOB FINISHED: status=FAILED exitCode=FAILED
|
||||
```
|
||||
|
||||
Full transcript: [`docs/output/11-resourceless-forgets-everything.txt`](output/11-resourceless-forgets-everything.txt).
|
||||
Run 2 does not throw `JobInstanceAlreadyCompleteException`, the exception you would expect for
|
||||
re-running an already-completed job (see [chapter 5](05-launching-and-jobparameters.md)). It does
|
||||
not know there was a run 1 at all. The resourceless `JobRepository` lives only in that JVM's
|
||||
heap; when the process exits, every `JobInstance`, `JobExecution`, and `StepExecution` it ever
|
||||
recorded goes with it. Run 2 starts `productImportJob` as if for the first time, tries to insert
|
||||
`ABC-0001` again, and collides with what run 1 already committed to the `PRODUCT` table —
|
||||
a table that, unlike the job repository, *is* backed by the same file-based H2 database both
|
||||
times.
|
||||
|
||||
This is the practical failure mode: **the job repository's persistence and your own application
|
||||
data's persistence are two separate decisions**, and it is entirely possible to get the second
|
||||
one right (a real database, a real table) while getting the first one wrong (resourceless,
|
||||
because `-jdbc` was left off a dependency list or excluded by a profile without realizing what it
|
||||
carried). The symptom is not a startup error. It is "my restart didn't restart" or "why did this
|
||||
job double-insert everything," discovered in whatever environment happens to restart the JVM
|
||||
between runs — which, for anything running in a container orchestrator, is any redeploy,
|
||||
crash-restart, or scale-down/scale-up cycle.
|
||||
|
||||
<blockquote style="background:#f4f5f7;border:1px solid #e2e5ea;border-left:4px solid #b7bec9;border-radius:6px;padding:16px 20px;"><strong>How to check which one you actually have.</strong> Query <code>information_schema.tables</code> (or your database's equivalent) for a table named <code>BATCH_JOB_INSTANCE</code> after your application starts. If it is not there, you have the resourceless job repository, whatever your dependency list looks like on paper — check for an exclusion in <code>spring.autoconfigure.exclude</code>, a conflicting <code>JobRepository</code> bean of your own, or simply a missing <code>-jdbc</code>/<code>-mongodb</code> starter.</blockquote>
|
||||
|
||||
## Going deeper
|
||||
|
||||
- `BatchProperties` and `BatchJdbcProperties`, including `spring.batch.job.enabled`,
|
||||
`spring.batch.jdbc.initialize-schema`, and `spring.batch.jdbc.table-prefix`: the
|
||||
`spring-configuration-metadata.json` inside `spring-boot-batch-4.1.1.jar` and
|
||||
`spring-boot-batch-jdbc-4.1.1.jar` is the authoritative list, more current than any blog post
|
||||
including this one.
|
||||
- MongoDB-backed job repositories follow the identical pattern via
|
||||
`spring-boot-starter-batch-data-mongodb`, not covered by this module.
|
||||
|
||||
[Next: Production checklist →](11-production-checklist.md)
|
||||
@@ -0,0 +1,47 @@
|
||||
# 11. Production checklist
|
||||
|
||||
[← Previous](10-resourceless-vs-jdbc.md) | [README](../README.md)
|
||||
|
||||
A short list, aimed at the gap between "this demo works" and "this is safe to point at a real
|
||||
feed." Each item links back to the chapter that explains the why.
|
||||
|
||||
- **Confirm the job repository is actually persistent** — query for `BATCH_JOB_INSTANCE`
|
||||
after startup, don't assume from the dependency list. [Chapter 10](10-resourceless-vs-jdbc.md).
|
||||
- **Pick chunk size from measurement, not habit.** Smaller costs more round trips; larger costs
|
||||
more rework per failure. [Chapter 3](03-chunk-oriented-processing.md).
|
||||
- **Decide skip vs. fail-and-restart per failure type, not per job.** A step can have a
|
||||
`skipLimit` for genuinely expected bad rows and still fail hard past that limit — the two
|
||||
are not mutually exclusive. [Chapter 8](08-skip-vs-restart.md).
|
||||
- **Use `chunk(int).transactionManager(tx)`, not `chunk(int, tx)`,** unless you specifically need
|
||||
the legacy `SimpleStepBuilder` for something the new model does not yet cover.
|
||||
[Chapter 2](02-anatomy-of-a-job.md).
|
||||
- **Do not use `beanMapped()` against a Java record** without checking your exact Spring
|
||||
Framework version's record support; prefer `itemPreparedStatementSetter` when in doubt.
|
||||
[Chapter 6](06-jdbc-writer-and-records.md).
|
||||
- **Give every step you want to always re-run on restart `allowStartIfComplete(true)`
|
||||
explicitly** — a reporting or notification step that silently gets skipped after a
|
||||
restart is a surprising, hard-to-notice gap. [Chapter 8](08-skip-vs-restart.md).
|
||||
- **A custom `ItemReader` that does not implement `ItemStream` restarts from scratch every time**,
|
||||
which is fine for an idempotent reader and a bug for anything else — check this
|
||||
deliberately rather than by finding out during an incident. [Chapter 7](07-restartability.md).
|
||||
- **`spring.batch.job.enabled=false` if you launch jobs yourself** through `JobOperator`, so the
|
||||
auto-configured runner does not also try. [Chapter 5](05-launching-and-jobparameters.md).
|
||||
- **Delete or secure any diagnostic endpoint you add while building this out.** This module has
|
||||
none, but the pattern (an endpoint dumping live `JobExplorer` state) is common enough to call
|
||||
out: it is invaluable while developing and a liability left in production.
|
||||
|
||||
## Should you even build a custom batch job for this?
|
||||
|
||||
Sometimes the honest answer is no. If the volume is small enough to fit in a single request-response
|
||||
cycle and does not need to survive a crash mid-way, a scheduled `@Component` method with its own
|
||||
try/catch is less machinery than a full Job/Step/JobRepository setup, and easier for the next
|
||||
person to read. Spring Batch earns its complexity when you actually need the things this article
|
||||
demonstrates — durable progress tracking, transactional chunking, a real restart story
|
||||
— not by default just because the word "batch" is in the requirements.
|
||||
|
||||
## Further reading
|
||||
|
||||
- [Spring Batch reference documentation](https://docs.spring.io/spring-batch/reference/) (`rel="nofollow"`)
|
||||
- [What's new in Spring Batch 6](https://docs.spring.io/spring-batch/reference/whatsnew.html) (`rel="nofollow"`)
|
||||
- [Spring Batch 6.0 Migration Guide](https://github.com/spring-projects/spring-batch/wiki/Spring-Batch-6.0-Migration-Guide) (`rel="nofollow"`, and see [chapter 9](09-corrections.md) for where this specific document was wrong)
|
||||
- [Spring Boot 4.1.0 release announcement](https://spring.io/blog/2026/06/10/spring-boot-4/) (`rel="nofollow"`)
|
||||
@@ -0,0 +1,10 @@
|
||||
# The processor as a filter: null means skip, not fail
|
||||
|
||||
input : Product[sku=ABC-0001, name=Widget 1, priceCents=1001]
|
||||
result : Product[sku=ABC-0001, name=Widget 1, priceCents=1001]
|
||||
|
||||
input : Product[sku=abc-0012, name=Widget 12, priceCents=1012] <- lowercase sku, fails ^[A-Z]{3}-\d{4}$
|
||||
result : null
|
||||
|
||||
input : Product[sku=ABC-0033, name=Widget 33, priceCents=0] <- priceCents is zero
|
||||
result : null
|
||||
@@ -0,0 +1,9 @@
|
||||
# Why skip(DataIntegrityViolationException.class) also catches the duplicate-key case
|
||||
|
||||
thrown type : org.springframework.dao.DuplicateKeyException
|
||||
is a DataIntegrityViolationException? true
|
||||
|
||||
H2's JdbcBatchUpdateException on a UNIQUE-constraint violation is translated by
|
||||
Spring's SQLExceptionSubclassTranslator into DuplicateKeyException, which extends
|
||||
DataIntegrityViolationException. A skip policy configured against the parent class
|
||||
catches the subclass too -- see docs/07-restartability.md and docs/08-skip-vs-restart.md.
|
||||
@@ -0,0 +1,19 @@
|
||||
# javap org.springframework.batch.core.step.builder.StepBuilder (spring-batch-core 6.0.5)
|
||||
|
||||
$ javap -cp spring-batch-core-6.0.5.jar org.springframework.batch.core.step.builder.StepBuilder
|
||||
Compiled from "StepBuilder.java"
|
||||
public class org.springframework.batch.core.step.builder.StepBuilder extends org.springframework.batch.core.step.builder.StepBuilderHelper<org.springframework.batch.core.step.builder.StepBuilder> {
|
||||
public org.springframework.batch.core.step.builder.StepBuilder(org.springframework.batch.core.repository.JobRepository);
|
||||
public org.springframework.batch.core.step.builder.StepBuilder(java.lang.String, org.springframework.batch.core.repository.JobRepository);
|
||||
public org.springframework.batch.core.step.builder.TaskletStepBuilder tasklet(org.springframework.batch.core.step.tasklet.Tasklet, org.springframework.transaction.PlatformTransactionManager);
|
||||
public org.springframework.batch.core.step.builder.TaskletStepBuilder tasklet(org.springframework.batch.core.step.tasklet.Tasklet);
|
||||
public <I, O> org.springframework.batch.core.step.builder.SimpleStepBuilder<I, O> chunk(int, org.springframework.transaction.PlatformTransactionManager);
|
||||
public <I, O> org.springframework.batch.core.step.builder.ChunkOrientedStepBuilder<I, O> chunk(int);
|
||||
public <I, O> org.springframework.batch.core.step.builder.SimpleStepBuilder<I, O> chunk(org.springframework.batch.infrastructure.repeat.CompletionPolicy, org.springframework.transaction.PlatformTransactionManager);
|
||||
public org.springframework.batch.core.step.builder.PartitionStepBuilder partitioner(java.lang.String, org.springframework.batch.core.partition.Partitioner);
|
||||
public org.springframework.batch.core.step.builder.PartitionStepBuilder partitioner(org.springframework.batch.core.step.Step);
|
||||
public org.springframework.batch.core.step.builder.JobStepBuilder job(org.springframework.batch.core.job.Job);
|
||||
public org.springframework.batch.core.step.builder.FlowStepBuilder flow(org.springframework.batch.core.job.flow.Flow);
|
||||
protected org.springframework.batch.core.step.builder.StepBuilder self();
|
||||
protected org.springframework.batch.core.step.builder.StepBuilderHelper self();
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
# javap: two ExecutionContext classes in two different packages
|
||||
|
||||
$ javap -cp spring-batch-infrastructure-6.0.5.jar org.springframework.batch.infrastructure.item.ExecutionContext
|
||||
Compiled from "ExecutionContext.java"
|
||||
public class org.springframework.batch.infrastructure.item.ExecutionContext implements java.io.Serializable {
|
||||
public org.springframework.batch.infrastructure.item.ExecutionContext();
|
||||
public org.springframework.batch.infrastructure.item.ExecutionContext(java.util.Map<java.lang.String, java.lang.Object>);
|
||||
public org.springframework.batch.infrastructure.item.ExecutionContext(org.springframework.batch.infrastructure.item.ExecutionContext);
|
||||
public void putString(java.lang.String, java.lang.String);
|
||||
public void putLong(java.lang.String, long);
|
||||
public void putInt(java.lang.String, int);
|
||||
public void putDouble(java.lang.String, double);
|
||||
public void put(java.lang.String, java.lang.Object);
|
||||
public boolean isDirty();
|
||||
public java.lang.String getString(java.lang.String);
|
||||
public java.lang.String getString(java.lang.String, java.lang.String);
|
||||
public long getLong(java.lang.String);
|
||||
public long getLong(java.lang.String, long);
|
||||
public int getInt(java.lang.String);
|
||||
public int getInt(java.lang.String, int);
|
||||
public double getDouble(java.lang.String);
|
||||
public double getDouble(java.lang.String, double);
|
||||
public java.lang.Object get(java.lang.String);
|
||||
public <V> V get(java.lang.String, java.lang.Class<V>);
|
||||
public <V> V get(java.lang.String, java.lang.Class<V>, V);
|
||||
public boolean isEmpty();
|
||||
public void clearDirtyFlag();
|
||||
public java.util.Set<java.util.Map$Entry<java.lang.String, java.lang.Object>> entrySet();
|
||||
public java.util.Map<java.lang.String, java.lang.Object> toMap();
|
||||
public boolean containsKey(java.lang.String);
|
||||
public java.lang.Object remove(java.lang.String);
|
||||
public boolean containsValue(java.lang.Object);
|
||||
public boolean equals(java.lang.Object);
|
||||
public int hashCode();
|
||||
public java.lang.String toString();
|
||||
public int size();
|
||||
}
|
||||
|
||||
$ javap -cp spring-batch-core-6.0.5.jar org.springframework.batch.core.repository.persistence.ExecutionContext
|
||||
Compiled from "ExecutionContext.java"
|
||||
public final class org.springframework.batch.core.repository.persistence.ExecutionContext extends java.lang.Record {
|
||||
public org.springframework.batch.core.repository.persistence.ExecutionContext(java.util.Map<java.lang.String, java.lang.Object>, boolean);
|
||||
public final java.lang.String toString();
|
||||
public final int hashCode();
|
||||
public final boolean equals(java.lang.Object);
|
||||
public java.util.Map<java.lang.String, java.lang.Object> map();
|
||||
public boolean dirty();
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
# The happy path: 60 rows in, 58 products out
|
||||
|
||||
$ java -jar target/spring-batch-1.0.0.jar --spring.profiles.active=clean \
|
||||
--import.file=file:scenario-data/clean-demo/input.csv \
|
||||
--spring.datasource.url=jdbc:h2:file:.../scenario-data/clean-demo/db
|
||||
|
||||
2026-09-13T06:23:05.509Z INFO 5242 --- [ main] o.s.batch.core.step.AbstractStep : Executing step: [importStep]
|
||||
2026-09-13T06:23:05.530Z WARN 5242 --- [ main] c.a.b.p.ProductValidatingProcessor : filtering abc-0012: does not match ^[A-Z]{3}-\d{4}$
|
||||
2026-09-13T06:23:05.543Z WARN 5242 --- [ main] c.a.b.p.ProductValidatingProcessor : filtering ABC-0033: priceCents must be positive, was 0
|
||||
2026-09-13T06:23:05.570Z INFO 5242 --- [ main] o.s.batch.core.step.AbstractStep : Step: [importStep] executed in 61ms
|
||||
2026-09-13T06:23:05.590Z INFO 5242 --- [ main] o.s.batch.core.step.AbstractStep : Executing step: [reportStep]
|
||||
REPORT: 58 products now in the PRODUCT table
|
||||
2026-09-13T06:23:05.600Z INFO 5242 --- [ main] o.s.batch.core.step.AbstractStep : Step: [reportStep] executed in 13ms
|
||||
JOB FINISHED: status=COMPLETED exitCode=COMPLETED
|
||||
|
||||
$ SELECT read_count, filter_count, write_count, commit_count FROM BATCH_STEP_EXECUTION;
|
||||
READ_COUNT | FILTER_COUNT | WRITE_COUNT | COMMIT_COUNT
|
||||
60 | 2 | 58 | 6
|
||||
(1 row, 21 ms)
|
||||
@@ -0,0 +1,23 @@
|
||||
# Run 1: the poisoned duplicate at row 47 fails the whole chunk
|
||||
|
||||
$ java -jar target/spring-batch-1.0.0.jar --spring.profiles.active=broken \
|
||||
--import.file=file:scenario-data/restart-demo/input.csv \
|
||||
--spring.datasource.url=jdbc:h2:file:.../scenario-data/restart-demo/db
|
||||
|
||||
2026-09-13T06:23:09.122Z INFO 5302 --- [ main] o.s.batch.core.step.AbstractStep : Executing step: [importStep]
|
||||
2026-09-13T06:23:09.162Z WARN 5302 --- [ main] c.a.b.p.ProductValidatingProcessor : filtering abc-0012: does not match ^[A-Z]{3}-\d{4}$
|
||||
2026-09-13T06:23:09.176Z WARN 5302 --- [ main] c.a.b.p.ProductValidatingProcessor : filtering ABC-0033: priceCents must be positive, was 0
|
||||
org.springframework.dao.DuplicateKeyException: PreparedStatementCallback; SQL [INSERT INTO PRODUCT (sku, name, price_cents) VALUES (?, ?, ?)]; Unique index or primary key violation: "PUBLIC.CONSTRAINT_18 INDEX PUBLIC.CONSTRAINT_INDEX_1 ON PUBLIC.PRODUCT(SKU NULLS FIRST) VALUES ( /* 5 */ 'ABC-0005' )"; SQL statement:
|
||||
Caused by: org.springframework.dao.DuplicateKeyException: PreparedStatementCallback; SQL [INSERT INTO PRODUCT (sku, name, price_cents) VALUES (?, ?, ?)]; Unique index or primary key violation: "PUBLIC.CONSTRAINT_18 INDEX PUBLIC.CONSTRAINT_INDEX_1 ON PUBLIC.PRODUCT(SKU NULLS FIRST) VALUES ( /* 5 */ 'ABC-0005' )"; SQL statement:
|
||||
2026-09-13T06:23:09.205Z INFO 5302 --- [ main] o.s.batch.core.step.AbstractStep : Step: [importStep] executed in 85ms
|
||||
JOB FINISHED: status=FAILED exitCode=FAILED
|
||||
|
||||
$ SELECT status, read_count, filter_count, write_count, commit_count, rollback_count FROM BATCH_STEP_EXECUTION;
|
||||
STATUS | READ_COUNT | FILTER_COUNT | WRITE_COUNT | COMMIT_COUNT | ROLLBACK_COUNT
|
||||
FAILED | 50 | 2 | 38 | 4 | 1
|
||||
(1 row, 21 ms)
|
||||
|
||||
$ SELECT COUNT(*) FROM PRODUCT;
|
||||
COUNT(*)
|
||||
38
|
||||
(1 row, 20 ms)
|
||||
@@ -0,0 +1,25 @@
|
||||
# Run 2: a brand-new JVM, the SAME job parameters, the corrected file
|
||||
|
||||
sed -i '48s/ABC-0005/ABC-0999/' scenario-data/restart-demo/input.csv # the only change
|
||||
|
||||
$ java -jar target/spring-batch-1.0.0.jar --spring.profiles.active=broken \
|
||||
--import.file=file:scenario-data/restart-demo/input.csv \
|
||||
--spring.datasource.url=jdbc:h2:file:.../scenario-data/restart-demo/db # same db file
|
||||
|
||||
2026-09-13T06:23:13.205Z INFO 5386 --- [ main] o.s.batch.core.step.AbstractStep : Executing step: [importStep]
|
||||
2026-09-13T06:23:13.232Z INFO 5386 --- [ main] o.s.batch.core.step.AbstractStep : Step: [importStep] executed in 28ms
|
||||
2026-09-13T06:23:13.240Z INFO 5386 --- [ main] o.s.batch.core.step.AbstractStep : Executing step: [reportStep]
|
||||
REPORT: 58 products now in the PRODUCT table
|
||||
2026-09-13T06:23:13.246Z INFO 5386 --- [ main] o.s.batch.core.step.AbstractStep : Step: [reportStep] executed in 6ms
|
||||
JOB FINISHED: status=COMPLETED exitCode=COMPLETED
|
||||
|
||||
$ SELECT step_execution_id, job_execution_id, status, read_count, write_count, commit_count, rollback_count FROM BATCH_STEP_EXECUTION ORDER BY step_execution_id;
|
||||
STEP_EXECUTION_ID | JOB_EXECUTION_ID | STATUS | READ_COUNT | WRITE_COUNT | COMMIT_COUNT | ROLLBACK_COUNT
|
||||
1 | 1 | FAILED | 50 | 38 | 4 | 1
|
||||
2 | 2 | COMPLETED | 20 | 20 | 2 | 0
|
||||
(2 rows, 22 ms)
|
||||
|
||||
$ SELECT COUNT(*) FROM PRODUCT;
|
||||
COUNT(*)
|
||||
58
|
||||
(1 row, 19 ms)
|
||||
@@ -0,0 +1,23 @@
|
||||
# javap: CommandLineJobRunner is deprecated (forRemoval), not removed, in 6.0.5
|
||||
|
||||
$ javap -verbose -cp spring-batch-core-6.0.5.jar org.springframework.batch.core.launch.support.CommandLineJobRunner | grep -A3 "^public class\|Deprecated"
|
||||
public class org.springframework.batch.core.launch.support.CommandLineJobRunner
|
||||
minor version: 0
|
||||
major version: 61
|
||||
flags: (0x0021) ACC_PUBLIC, ACC_SUPER
|
||||
--
|
||||
#503 = Utf8 Deprecated
|
||||
#504 = Utf8 RuntimeVisibleAnnotations
|
||||
#505 = Utf8 Ljava/lang/Deprecated;
|
||||
#506 = Utf8 since
|
||||
#507 = Utf8 6.0
|
||||
#508 = Utf8 forRemoval
|
||||
--
|
||||
Deprecated: true
|
||||
RuntimeVisibleAnnotations:
|
||||
0: #505(#506=s#507,#508=Z#509)
|
||||
java.lang.Deprecated(
|
||||
since="6.0"
|
||||
forRemoval=true
|
||||
)
|
||||
--
|
||||
@@ -0,0 +1,27 @@
|
||||
# faultTolerant().skip(DataIntegrityViolationException.class): one bad row, job still COMPLETES
|
||||
|
||||
$ java -jar target/spring-batch-1.0.0.jar --spring.profiles.active=skip \
|
||||
--import.file=file:scenario-data/skip-demo/input.csv \
|
||||
--spring.datasource.url=jdbc:h2:file:.../scenario-data/skip-demo/db
|
||||
|
||||
2026-09-13T06:23:17.111Z INFO 5469 --- [ main] o.s.batch.core.step.AbstractStep : Executing step: [importStep]
|
||||
2026-09-13T06:23:17.139Z WARN 5469 --- [ main] c.a.b.p.ProductValidatingProcessor : filtering abc-0012: does not match ^[A-Z]{3}-\d{4}$
|
||||
2026-09-13T06:23:17.150Z WARN 5469 --- [ main] c.a.b.p.ProductValidatingProcessor : filtering ABC-0033: priceCents must be positive, was 0
|
||||
Caused by: org.springframework.dao.DuplicateKeyException: PreparedStatementCallback; SQL [INSERT INTO PRODUCT (sku, name, price_cents) VALUES (?, ?, ?)]; Unique index or primary key violation: "PUBLIC.CONSTRAINT_18 INDEX PUBLIC.CONSTRAINT_INDEX_1 ON PUBLIC.PRODUCT(SKU NULLS FIRST) VALUES ( /* 5 */ 'ABC-0005' )"; SQL statement:
|
||||
Caused by: org.springframework.dao.DuplicateKeyException: PreparedStatementCallback; SQL [INSERT INTO PRODUCT (sku, name, price_cents) VALUES (?, ?, ?)]; Unique index or primary key violation: "PUBLIC.CONSTRAINT_18 INDEX PUBLIC.CONSTRAINT_INDEX_1 ON PUBLIC.PRODUCT(SKU NULLS FIRST) VALUES ( /* 5 */ 'ABC-0005' )"; SQL statement:
|
||||
SKIPPED on write: ABC-0005 (DuplicateKeyException: PreparedStatementCallback; SQL [INSERT INTO PRODUCT (sku, name, price_cents) VALUES (?, ?, ?)]; Unique index or primary key violation: "PUBLIC.CONSTRAINT_18 INDEX PUBLIC.CONSTRAINT_INDEX_1 ON PUBLIC.PRODUCT(SKU NULLS FIRST) VALUES ( /* 5 */ 'ABC-0005' )"; SQL statement:
|
||||
2026-09-13T06:23:17.235Z INFO 5469 --- [ main] o.s.batch.core.step.AbstractStep : Step: [importStep] executed in 124ms
|
||||
2026-09-13T06:23:17.247Z INFO 5469 --- [ main] o.s.batch.core.step.AbstractStep : Executing step: [reportStep]
|
||||
REPORT: 57 products now in the PRODUCT table
|
||||
2026-09-13T06:23:17.256Z INFO 5469 --- [ main] o.s.batch.core.step.AbstractStep : Step: [reportStep] executed in 9ms
|
||||
JOB FINISHED: status=COMPLETED exitCode=COMPLETED
|
||||
|
||||
$ SELECT status, read_count, filter_count, write_count, write_skip_count, commit_count, rollback_count FROM BATCH_STEP_EXECUTION;
|
||||
STATUS | READ_COUNT | FILTER_COUNT | WRITE_COUNT | WRITE_SKIP_COUNT | COMMIT_COUNT | ROLLBACK_COUNT
|
||||
COMPLETED | 60 | 2 | 57 | 1 | 6 | 2
|
||||
(1 row, 22 ms)
|
||||
|
||||
$ SELECT COUNT(*) FROM PRODUCT;
|
||||
COUNT(*)
|
||||
57
|
||||
(1 row, 19 ms)
|
||||
@@ -0,0 +1,21 @@
|
||||
# --spring.autoconfigure.exclude=...BatchJdbcAutoConfiguration: same job, same params, no memory
|
||||
|
||||
$ java -jar target/spring-batch-1.0.0.jar --spring.profiles.active=clean \
|
||||
--spring.autoconfigure.exclude=org.springframework.boot.batch.jdbc.autoconfigure.BatchJdbcAutoConfiguration \
|
||||
--import.file=file:scenario-data/resourceless-demo/input.csv \
|
||||
--spring.datasource.url=jdbc:h2:file:.../scenario-data/resourceless-demo/db
|
||||
|
||||
-- run 1 --
|
||||
REPORT: 58 products now in the PRODUCT table
|
||||
JOB FINISHED: status=COMPLETED exitCode=COMPLETED
|
||||
|
||||
-- run 2: a second, completely fresh JVM, same jar, same job parameters, same PRODUCT table --
|
||||
org.springframework.dao.DuplicateKeyException: PreparedStatementCallback; SQL [INSERT INTO PRODUCT (sku, name, price_cents) VALUES (?, ?, ?)]; Unique index or primary key violation: "PUBLIC.CONSTRAINT_18 INDEX PUBLIC.CONSTRAINT_INDEX_1 ON PUBLIC.PRODUCT(SKU NULLS FIRST) VALUES ( /* 1 */ 'ABC-0001' )"; SQL statement:
|
||||
Caused by: org.springframework.dao.DuplicateKeyException: PreparedStatementCallback; SQL [INSERT INTO PRODUCT (sku, name, price_cents) VALUES (?, ?, ?)]; Unique index or primary key violation: "PUBLIC.CONSTRAINT_18 INDEX PUBLIC.CONSTRAINT_INDEX_1 ON PUBLIC.PRODUCT(SKU NULLS FIRST) VALUES ( /* 1 */ 'ABC-0001' )"; SQL statement:
|
||||
JOB FINISHED: status=FAILED exitCode=FAILED
|
||||
|
||||
No JobInstanceAlreadyCompleteException on run 2 -- the resourceless JobRepository has no idea
|
||||
run 1 ever happened, so it tries the whole job again and collides with what run 1 already
|
||||
wrote to the PRODUCT table. With spring-boot-starter-batch-jdbc (the default configuration
|
||||
used everywhere else in this module) run 2 throws JobInstanceAlreadyCompleteException instead,
|
||||
which ImportRunner treats as "nothing to do" -- see docs/07-restartability.md.
|
||||
Reference in New Issue
Block a user