spring-boot-demo/spring-batch/docs/03-chunk-oriented-processing.md

# 3. Chunk-oriented processing: what "chunk(10)" actually does

[← Previous](02-anatomy-of-a-job.md) | [README](../README.md) | [Next: The item processor as a filter →](04-item-processor-as-filter.md)

`chunk(10)` means: read up to 10 items (calling the reader once per item), run each through the
processor, then hand all the survivors to the writer in **one call**, inside **one transaction**.
If the writer succeeds, the transaction commits and the step's counters — read, write,
filter, commit — move forward together. If anything in that chunk throws, the whole
transaction rolls back: none of those 10 items' writes are kept, even the ones that would have
succeeded on their own.

The happy-path run of this module's `productImportJob` against clean data makes this concrete.
60 rows, chunk size 10, two rows fail the processor's validation (filtered, not written):

```console
$ SELECT read_count, filter_count, write_count, commit_count FROM BATCH_STEP_EXECUTION;
READ_COUNT | FILTER_COUNT | WRITE_COUNT | COMMIT_COUNT
60         | 2            | 58          | 6
```

Full transcript: [`docs/output/05-happy-path.txt`](output/05-happy-path.txt), source:
[`ProductValidatingProcessor.java`](../src/main/java/com/ankurm/batch/processing/ProductValidatingProcessor.java).
60 rows at chunk size 10 is 6 chunks — `COMMIT_COUNT` confirms all 6 committed, one
transaction each.

<figure>
<svg viewBox="0 0 740 220" role="img" aria-label="60 rows split into six chunks of 10; each chunk is read, processed, and written as one transaction that either commits fully or rolls back fully.">
<style>.h{font:600 12px sans-serif;fill:#1a1a1a}.c{font:11px sans-serif;fill:#4b5563}.m{font:11px monospace;fill:#1a1a1a}</style>
<text x="20" y="24" class="h">60 rows &rarr; six chunks of 10, each its own transaction</text>
<g>
<!-- 6 chunk boxes -->
<rect x="20"  y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
<rect x="140" y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
<rect x="260" y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
<rect x="380" y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
<rect x="500" y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
<rect x="620" y="50" width="110" height="50" rx="5" fill="#e7f4ea" stroke="#4a9d63"/>
<text x="45"  y="80" class="m">1-10</text>
<text x="165" y="80" class="m">11-20</text>
<text x="285" y="80" class="m">21-30</text>
<text x="405" y="80" class="m">31-40</text>
<text x="525" y="80" class="m">41-50</text>
<text x="645" y="80" class="m">51-60</text>
</g>
<text x="20" y="130" class="c">Each box: read 10 &rarr; process 10 (0-1 filtered) &rarr; write survivors &rarr; COMMIT.</text>
<text x="20" y="150" class="c">Row 12 (chunk 2) and row 33 (chunk 4) are filtered by the processor -- see chapter 4 -- so those</text>
<text x="20" y="170" class="c">two chunks commit 9 rows instead of 10. 58 rows land in PRODUCT; every chunk still commits.</text>
</svg>
</figure>

## Why 10, and what changes if you pick a different size

Chunk size is a trade-off with no universally right answer:

- **Smaller chunks** commit more often, so a failure loses less work and transactions are
  shorter (less lock contention, smaller rollback). They also mean more round trips to the
  database &mdash; more commit overhead per row.
- **Larger chunks** amortize that overhead across more rows, but a failure anywhere in the chunk
  costs you the whole chunk's work, and the transaction held open is bigger and longer.

10 is small enough that chapter 7's poisoned row costs you 12 rows of rework at most (one
chunk), not thousands, and small enough to demonstrate multiple commits from 60 rows without a
huge fixture file. Production jobs processing millions of rows commonly use chunk sizes in the
hundreds or low thousands; the right number depends on row size, write cost, and how expensive a
partial redo is for your specific job &mdash; there is no formula that replaces measuring it
against your own writer.

## Going deeper

- The chunk-processing reference: [Spring Batch &mdash; Chunk-oriented processing](https://docs.spring.io/spring-batch/reference/step/chunk-oriented-processing.html) (`rel="nofollow"`).
- `ChunkOrientedStep` itself, if you want to see the commit/rollback logic this chapter
  describes: `org.springframework.batch.core.step.item.ChunkOrientedStep` in
  `spring-batch-core-6.0.5.jar` (the stack trace in
  [`docs/output/07-restart-run1-fails.txt`](output/07-restart-run1-fails.txt) names it directly).
- What happens when the processor itself decides to drop a row rather than the writer failing:
  [chapter 4](04-item-processor-as-filter.md).

[Next: The item processor as a filter &rarr;](04-item-processor-as-filter.md)