Spring AI + Spring Batch: Classifying 100,000 Support Tickets with an LLM (Throttling, Retries, Cost)

You have 100,000 support tickets in a table and a model that can sort each one into BILLING, TECHNICAL, ACCOUNT, SHIPPING, FEATURE_REQUEST or OTHER. The loop is four lines long. Then reality turns up: the provider answers 429 Too Many Requests after the first few hundred calls, a few replies in a hundred are not JSON, the job dies at ticket 40,000 and somebody asks two questions at once — how much did that cost? and can we just run it again?

Spring Batch already has good answers to the second question, and I wrote about them in Spring Batch on Boot 4.1: Jobs, Steps, Chunk Processing and Restartability. Spring AI answers the first half of the problem: a ChatClient that turns a ticket into a reply and tells you how many tokens it used. This article puts the two together and then does the unglamorous part: it makes the job slow down when the provider says no, stop paying for the same bad answer twice, skip what cannot be fixed and write down why, keep a cost ledger that is committed with the data it paid for, and survive kill -9 halfway through.

Everything below ran. The job processes 100,000 generated tickets in about a minute on a shared 2-vCPU sandbox, and it can do that because the model is a deterministic simulation, not a language model: no provider was called, no key exists, and nothing here says anything about the accuracy, latency or price of a real model. What it does let you do is break the pipeline in exactly reproducible ways — 429s, 503s, truncated JSON, a model regression, a crash — and watch every counter.

Versions used throughout. Spring Boot 4.1.2, which manages Spring Batch 6.0.6 and Spring Framework 7.0.10; Spring AI 2.0.2; Resilience4j 2.4.0 via resilience4j-spring-boot4; Java 25 (Temurin 25.0.5); H2 2.4.240. Read from the running classpath, not assumed: output/01-versions.txt. Spring Batch 6.1.0-RC1 exists on Maven Central but is not GA, so nothing here uses it. Prices in this article are configurable placeholders, not the price of any provider or model.

A batch job that calls an LLM is a batch job whose worker costs money and sometimes says no

If you have not used Spring Batch: a job is made of steps, and a chunk-oriented step repeats read one item, process it, and after N items write them all in one transaction. The reader remembers its position in the database, inside the same transaction as the writes. That one design decision is why a crashed job can resume instead of starting over. If you have not used Spring AI’s ChatClient: it is a thin fluent client over a ChatModel; you give it a prompt and get back a ChatResponse whose metadata carries a Usage object with prompt and completion token counts.

Here is the whole pipeline. The processor is where all the new problems live, because it is the only part of the step that talks to something outside your control.

The diagram makes one promise worth holding on to: the ledger row is in the same transaction as the tickets it paid for. If a chunk commits, its money is recorded; if it rolls back, it is not. Later in the article this turns out to be the only way to answer “what did the crash cost us?” with a number.

What “the model” is in this repository. SimulatedProvider is a hand-written ChatModel, the same idea as the ScriptedChatModel used in the other modules of the companion repository, but driven by a seeded fault plan instead of a list of replies. It reads Ticket #<id> out of the prompt and, from a hash of (seed, ticket, call number), returns valid JSON, JSON in a markdown fence, truncated JSON, prose before the JSON, a category that is not in the schema, a missing field, a 429, a 503 or a slow answer. It enforces its own calls-per-second quota, and it estimates tokens as characters / 4. Swap that one bean for a real ChatModel and nothing else changes; I could not run that variant here and did not pretend to. The fault mix it produces is in output/02-simulated-provider.txt.

Outcome of the first call per ticket (the plan: 1.5% 429, 0.5% 503, 0.3% never valid, 1.2% invalid on the first reply only):
  429                                                  304   1.52%
  503                                                   93   0.47%
  bad JSON: category not in schema                      76   0.38%
  bad JSON: missing field                               77   0.39%
  bad JSON: not JSON                                   137   0.69%
  valid JSON                                        17,361  86.81%
  valid JSON (in a markdown fence, stripped)         1,952   9.76%

Output: output/02-simulated-provider.txt. Two separate provider instances with the same seed produce identical outcome lists over 20,000 tickets, which is what makes every number in this article reproducible. These percentages describe the test plan, not any real service.

The smallest version: one processor, one writer, a fault-tolerant chunk step

The processor is an ordinary ItemProcessor<Ticket, Classification>. Its core is a loop that asks the model, records the usage from the response metadata, and tries to parse the answer. Source: src/main/java/com/ankurm/batchllm/TicketClassifier.java.

    @Override
    public Classification process(Ticket ticket) {
        BadModelOutputException last = null;
        for (int ask = 0; ask <= props.reask(); ask++) {
            ChatResponse response = callModel(ticket);
            Usage usage = response.getMetadata().getUsage();
            meter.billedCalls.incrementAndGet();
            meter.promptTokens.addAndGet(usage.getPromptTokens());
            meter.completionTokens.addAndGet(usage.getCompletionTokens());
            String text = response.getResult().getOutput().getText();
            try {
                return parse(ticket.id(), text);
            } catch (BadModelOutputException e) {
                meter.badReplies.incrementAndGet();
                last = e;
            }
        }
        meter.skipped.incrementAndGet();
        throw last;
    }

Two details to notice. Usage is recorded for every reply, including the ones that turn out to be unusable, because the provider bills for them. And after the allowed re-asks the processor throws; it does not return null. In Spring Batch, returning null from a processor means “filter this item out” and is counted separately; throwing is what the skip policy sees.

The step is built with the Spring Batch 6 ChunkOrientedStepBuilder. Source: src/main/java/com/ankurm/batchllm/LlmJobConfig.java.

        ChunkOrientedStepBuilder<Ticket, Classification> step = new StepBuilder("classifyStep", jobRepository)
                .<Ticket, Classification>chunk(props.chunkSize())
                .transactionManager(transactionManager)
                .reader(reader)
                .processor(classifier)
                .writer(writer)
                .taskExecutor(llmExecutor)
                .faultTolerant()
                // Deterministic bad output: skip it, remember it, keep going -- up to a limit.
                .skip(BadModelOutputException.class)
                // ... and a row the database refuses (too long, constraint) is just as deterministic.
                .skip(org.springframework.dao.DataIntegrityViolationException.class)
                .skipLimit(props.skipLimit())
                .skipListener(skipListener);
        if (!props.processorTransactional()) {
            step.processorNonTransactional();
        }

Run end to end over all 100,000 generated tickets, from a freshly built jar, a fresh H2 file and a fresh provider meter (scripts/run-clean.sh): the job completed in 55.2 seconds on a shared sandbox (indicative; other processes were using the same two cores). Output: output/08-clean-run.txt.

START run.id=clean tickets=100000 chunk=100
JOB FINISHED: executionId=1 status=COMPLETED seconds=55.2
---- REPORT ----
tickets total=100000 classified=99691 skipped=309 unaccounted=0
step executions:
  {ID=1, JOB=1, STATUS=COMPLETED, READS=100000, WRITES=99691, SKIPS=309, COMMITS=1000, ROLLBACKS=0}
ledger (committed chunks only): {ROWS_=1000, WRITTEN=99691, SKIPPED=309, ATTEMPTS=103519, R429=1531, U503=458, RETRIES=1989, BILLED=101530, BAD=1839, PT=10809111, CT=1467192, COST=7.605343500}

Every ticket is accounted for: 99,691 classified plus 309 skipped is 100,000, and unaccounted=0 is a number the report computes, not one I typed. The step execution row shows COMMITS=1000, which is 100,000 / 100, and ROLLBACKS=0.

When the provider says 429: take a permit before every attempt, and retry outside the limiter

A rate limiter answers “may I start a call now?” by handing out a fixed number of permits per time window and making you wait when they run out. A retry answers “that failed for a temporary reason; try again after a pause.” They are different tools, and the order you stack them in matters: if the limiter is outside the retry, a retried attempt does not take a new permit, so a burst of retries can exceed the quota you were trying to respect.

The processor stacks them as Retry( RateLimiter( call ) ) — every attempt, first or fifth, goes through the limiter. The same file, src/main/java/com/ankurm/batchllm/TicketClassifier.java:

    private ChatResponse callModel(Ticket ticket) {
        Supplier<ChatResponse> attempt = () -> {
            meter.attempts.incrementAndGet();
            try {
                return chatClient.prompt()
                        .user("Ticket #" + ticket.id() + "\nSubject: " + ticket.subject() + "\nBody: " + ticket.body())
                        .call()
                        .chatResponse();
            } catch (ProviderRateLimitException e) {
                meter.rejected429.incrementAndGet();
                throw e;
            } catch (ProviderUnavailableException e) {
                meter.unavailable503.incrementAndGet();
                throw e;
            }
        };
        // Retry( RateLimiter( call ) ): every retry goes back through the limiter.
        return Retry.decorateSupplier(retry, RateLimiter.decorateSupplier(limiter, attempt)).get();
    }

The diagram is the argument: the order of the two wrappers decides whether retries are counted against your quota. The configuration is plain Resilience4j, read by resilience4j-spring-boot4. Source: src/main/resources/application.yml.

# The model call's two Resilience4j policies. The scale of time here is deliberately small: see README.
resilience4j:
  ratelimiter:
    instances:
      llm:
        limit-for-period: 200
        limit-refresh-period: 100ms
        timeout-duration: 60s
  retry:
    instances:
      llm:
        max-attempts: 4
        wait-duration: 25ms
        enable-exponential-backoff: true
        exponential-backoff-multiplier: 2
        retry-exceptions:
          - com.ankurm.batchllm.ProviderRateLimitException
          - com.ankurm.batchllm.ProviderUnavailableException

Now the experiment. The same 4,000 tickets, the same fault plan, 64 threads, against a simulated provider whose quota is 800 calls per second. The first run has no limiter (a permit allowance so large it never blocks); the second allows 60 permits per 100 ms, which is 600 per second. Output: output/03-throttle.txt.

scenario           ok bad JSON  exhausted     calls  quota 429s  retries  seconds
no limiter       3066       13        921     7,596       4,396    3,543      3.5
limiter 600/s    3986       14          0     4,146           0       85      6.9

Without a limiter the client exceeded the quota 4,396 times, burned 3,543 retries, and 921 tickets exhausted all four attempts — in a real step each of those is a failed item, and without a skip policy for it a failed step. With the limiter the provider received 4,146 calls, rejected none for quota reasons (quota 429s = 0) and every ticket got an answer. The limited run took longer (6.9 s against 3.5 s); that is the price of not being thrown out, and the unlimited run’s time is not a fair comparison anyway because 921 of its tickets simply failed. Times are indicative: shared 2-vCPU sandbox.

How the 100,000-ticket run is scaled, honestly. Real quotas are far lower than the simulation’s. The job’s limiter allows 200 permits per 100 ms (2,000 calls per second), the simulated provider’s own quota is 2,400 per second, and a simulated call takes 1–3 ms (1% take 40 ms). The clean run made 103,519 attempts in 55.2 s, about 1,875 per second, which is the limiter’s ceiling doing its job. Against a real provider that allows, say, 50 requests per second (a number I chose for the arithmetic, not a fact about anyone), the same 103,519 attempts take at least 103,519 / 50 = 2,070 s, about 35 minutes, however many threads you add. The shape is the same; only the clock is scaled by roughly 40.

When the model answers but the answer is not JSON: skip it, write down why, and re-ask once

A reply that arrives and does not parse is a different kind of failure from a 429. A 429 is about when you asked; a bad reply is about what came back, and asking again at the same temperature with the same prompt will often give the same answer while billing you for it. So the rule here has three parts: validate strictly (JSON, all three fields, category in the enum, priority 1–4), allow one re-ask because a small share of bad replies are one-off (reask: 1), and after that throw a BadModelOutputException that the step’s skip policy turns into a recorded skip.

The skip listener writes the ticket, the reason and the raw reply to a table, so a skipped ticket is a row you can re-run later and not just a counter. Source: src/main/java/com/ankurm/batchllm/SkippedTicketListener.java. A 3,000-ticket run through the real job shows the first skipped ticket of each kind. Output: output/04-skip-and-ledger.txt.

job status                     COMPLETED
step: read=3000 write=2987 processSkip=13 commits=30 rollbacks=0
TICKET_RESULT rows             2987
SKIPPED_TICKET rows            13
classified + skipped           3000   (tickets: 3000)

Skipped tickets and why (first one of each kind):
  ticket 46    category not in schema   raw reply: {"category":"Account & Payments","priority":1,"confidence":0.74}
  ticket 234   not JSON                 raw reply: Sure! Here is the classification you asked for: {"category":"BILLING","priority":4,"confidence":0.7}
  ticket 491   missing field            raw reply: {"category":"OTHER","priority":2}

The numbers reconcile, and the reconciliation is worth doing yourself because it catches bugs. For the 100,000-ticket run: billed calls 101,530 = 100,000 first asks + 1,530 re-asks. The 1,530 re-asks split into 309 tickets that were bad twice (skipped) and 1,221 that were bad once and fine on the second reply, which is why 1,839 bad replies (1.8% of billed calls) produced only 309 skips: 1,221 + 2 × 309 = 1,839. The provider’s attempts, 103,519, are the 101,530 billed calls plus 1,531 quota or injected 429s plus 458 injected 503s (101,530 + 1,531 + 458 = 103,519). All of those come from output/08-clean-run.txt.

A bug the ticket count found. My first skip listener inserted the skipped ticket with a plain JdbcTemplate. For a skip during processing that is fine. For a skip during writing, Spring Batch calls onSkipInWrite while the chunk transaction is about to be rolled back — so the insert joined it and vanished with it. The symptom was 299 tickets accounted for out of 300 in a test that asserts classified + skipped == total. The fix is the REQUIRES_NEW transaction template in the listener (the file is linked above). If you take one habit from this article, take that assertion.

Execution 1: simulated model returns invalid JSON for 50% of tickets, no re-ask, skip limit 50, 2,000 tickets.
  status / reason         FAILED / org.springframework.batch.core.step.FatalStepExecutionException: Unable to process chunk
  tickets read            200 of 2000
  calls the provider billed before the step stopped: 200  (a full run needs about 2,000)
  committed results       52     skipped rows 50

Execution 2: same job parameters, same database, model fixed, skip limit 1000.
  status                  COMPLETED
  tickets read this time  1900    (execution 1 had committed 100, so the reader resumed after them)
  committed results       1942     skipped rows 58     classified + skipped = 2000
  tickets present in BOTH tables: 0  (stale skip rows from execution 1 are deleted when the ticket is classified)

Two retry mechanisms, one failure: where Resilience4j belongs and where Spring Batch’s does

Spring Batch 6 can retry too. Call faultTolerant() and give the builder a retryPolicy(...) (a Spring Framework 7 org.springframework.core.retry.RetryPolicy, not the older spring-retry class) and the chunk step retries failed process and write operations. That raises an obvious question: why have Resilience4j at all? The answer is about what each one can see.

Table of my recommendation, not of a rule in either project. The experiments below are what back it. All three count calls at the provider’s meter, because that is where the money is. Output: output/05-retry-layers.txt.

(a) A transient DB failure on the write of chunk 2 (tickets 101-200), 300 tickets, Spring Batch write retry = 2
    no failure                   status=COMPLETED billed calls=304  rollbacks=0
    failure + Spring Batch retry status=COMPLETED billed calls=304  rollbacks=0
    -> the write was retried inside the same transaction; the model was not asked again.

(b) The provider answers 429 to everything, 1 ticket. Calls the provider received for that ticket:
    Resilience4j retry only (max 4 attempts)     status=FAILED  attempts=4
    + Spring Batch retry of the same exception (2) status=FAILED  attempts=12
    -> 4 x (1 + 2) = 12. Two retry layers on the same failure multiply, and each attempt also takes a limiter permit.

(c) A poison row at WRITE time (ticket 250): chunk 3 (201-300) rolls back and is scanned one item at a time.
    processorNonTransactional (cached)       status=COMPLETED billed calls=304  (no-failure baseline: 304)  rollbacks=2  ledger rows=100  skipped=3
    default (processor runs again per item)  status=COMPLETED billed calls=403  (extra over baseline: 99)  rollbacks=2  ledger rows=100  skipped=3
    -> without processorNonTransactional(), every item of the failed chunk is classified a second time and paid for twice.

Read the three experiments in order. (a) A transient database failure on one chunk write, with Spring Batch’s retry enabled: the job completed, the provider’s billed-call count is identical to the no-failure run (304), and no rollback was recorded — the write was retried without asking the model again. (b) The provider answers 429 to everything. With Resilience4j alone, one ticket produces 4 attempts. Add Spring Batch’s retry for the same exception and it produces 12: four Resilience4j attempts inside each of three Spring Batch attempts. Two retry layers on one failure multiply, and every one of those attempts also spends a limiter permit. (c) A poison row at write time makes the chunk roll back and be re-attempted one item at a time (that is the scan). By default Spring Batch re-runs the processor for each of those items — here 99 extra billed calls, 403 against 304. processorNonTransactional(), added in Spring Batch 6.0.6, reuses the cached processor output instead and bills 304.

The trap in (c). Nothing fails and nothing logs a warning. The job completes, the data is right, and the bill for any chunk that hits a write error is doubled. The fingerprint is a ledger whose billed calls exceed tickets plus re-asks. The setting is on for this module’s step (see src/main/java/com/ankurm/batchllm/LlmJobConfig.java); the reason the default is the other way is that a processor whose output a failed write has mutated (a JPA entity, say) must be re-run. Ours returns an immutable record, so caching it is safe.

The cost ledger: one row per committed chunk, in the same transaction as the tickets it paid for

Tokens are the measurement; money is tokens multiplied by a price table you supply. Each processed ticket adds its response’s Usage to an in-memory meter, and the writer drains that meter into a COST_LEDGER row inside the chunk transaction. If the transaction rolls back, a transaction synchronization puts the amounts back — the calls were still made and billed — so they land in the next row. Source: src/main/java/com/ankurm/batchllm/ClassificationWriter.java.

        // A ticket skipped in an earlier, uncommitted attempt (the skip listener commits on its own) and
        // classified now must not stay in the skipped table.
        jdbc.batchUpdate("DELETE FROM SKIPPED_TICKET WHERE TICKET_ID = ?", items, items.size(),
                (ps, c) -> ps.setLong(1, c.ticketId()));

        long first = items.stream().mapToLong(Classification::ticketId).min().orElse(0);
        long last = items.stream().mapToLong(Classification::ticketId).max().orElse(0);
        UsageMeter.Snapshot spent = meter.drain();
        if (TransactionSynchronizationManager.isSynchronizationActive()) {
            TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronization() {
                @Override
                public void afterCompletion(int status) {
                    if (status == STATUS_COMMITTED) {
                        long rows = committedLedgerRows.incrementAndGet();
                        if (rows % 50 == 0) {
                            System.out.println("PROGRESS committed-chunks-this-jvm=" + rows + " last-ticket=" + last);
                        }
                    } else {
                        meter.restore(spent); // rolled back: the money was still spent
                    }
                }
            });
        }
        BigDecimal cost = pricing.cost(spent.promptTokens(), spent.completionTokens());
        jdbc.update("""
                INSERT INTO COST_LEDGER (FIRST_TICKET, LAST_TICKET, TICKETS_WRITTEN, TICKETS_SKIPPED, ATTEMPTS,
                  REJECTED_429, UNAVAILABLE_503, RETRIES, BILLED_CALLS, BAD_REPLIES, PROMPT_TOKENS, COMPLETION_TOKENS,
                  COST, PRICE_VERSION) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?)""",
                first, last, items.size(), spent.skipped(), spent.attempts(), spent.rejected429(),
                spent.unavailable503(), spent.retries(), spent.billedCalls(), spent.badReplies(),
                spent.promptTokens(), spent.completionTokens(), cost, pricing.version());

The prices are an ordinary configuration record, deliberately named so nobody mistakes them for facts. Source: src/main/java/com/ankurm/batchllm/Pricing.java.

@ConfigurationProperties("batch.pricing")
public record Pricing(
        @DefaultValue("EXAMPLE-NOT-A-REAL-PRICE") String version,
        @DefaultValue("0.50") BigDecimal inputPerMillion,
        @DefaultValue("1.50") BigDecimal outputPerMillion) {

    private static final BigDecimal MILLION = BigDecimal.valueOf(1_000_000L);

    public BigDecimal cost(long promptTokens, long completionTokens) {
        return inputPerMillion.multiply(BigDecimal.valueOf(promptTokens))
                .add(outputPerMillion.multiply(BigDecimal.valueOf(completionTokens)))
                .divide(MILLION, 9, RoundingMode.HALF_UP);
    }

Proof that the ledger is honest comes from a second, independent counter. The simulated provider keeps its own invoice in a memory-mapped file, and the report compares the two. For the 3,000-ticket run through the real job, output output/04-skip-and-ledger.txt:

First 3 ledger rows (one per committed chunk):
  tickets      written skipped attempts   429s   503s    bad promptTok complTok         cost
  1-100             99       1      104      1      1      3     10831     1469  0.007619000
  101-200          100       0      100      0      0      0     10574     1453  0.007466500
  201-300           99       1      105      2      1      3     10736     1495  0.007610500

Ledger totals vs the provider's own meter (they must agree when nothing was lost):
                               ledger     provider
  calls received                3,104        3,104
  429 rejections                   39           39
  503 rejections                   14           14
  billed calls                  3,051        3,051
  prompt tokens               324,790      324,790
  completion tokens            44,107       44,107

Cost = prompt tokens x input price + completion tokens x output price, per million tokens.
  table 'EXAMPLE-NOT-A-REAL-PRICE' (input 0.50, output 1.50): 0.228555500   <- the ledger's total
  same tokens under table 'OTHER-EXAMPLE' (input 2.00, output 6.00):  0.914222000
  Both tables are invented inputs. Tokens are the measurement; money is tokens x a table you supply.
  Tokens spent on replies that were then skipped: bad replies billed = 64 of 3051 billed calls.

Every line agrees. And the cost line is where the label matters: the same token counts cost 0.228555500 under the placeholder table and 0.914222000 under a second invented one. The tokens did not change; only the table did, and each ledger row stores the table’s PRICE_VERSION so a price change shows up in the data instead of silently rewriting history. For the full 100,000 the ledger totals 10,809,111 prompt tokens and 1,467,192 completion tokens, which is 7.6053435 in placeholder units (output/08-clean-run.txt).

kill -9 at chunk 400: a second JVM finishes the job and the result is byte-identical

Restart in Spring Batch is not a feature you switch on; it falls out of three facts. The job repository is in the database. The job is identified by its parameters (run.id=crash-demo), so starting it again means “this instance”. And the reader saves its position in the step’s ExecutionContext, which Spring Batch writes inside the same transaction as the chunk (I confirmed that ordering in saveState in the 6.0.6 source). The reader is short: src/main/java/com/ankurm/batchllm/TicketReader.java.

The demo is two real java -jar processes, no test harness. scripts/crash-restart.sh starts JVM 1, watches its output for the 400th committed chunk, sends kill -9, then starts JVM 2 with the same parameters and the same H2 file. JVM 2 finds an execution still marked STARTED whose process no longer exists; an execution still marked STARTED counts as running, so the runner calls JobOperator.recover(...). Its Javadoc (since 6.0) says it marks an execution stuck in STARTED after an abrupt shutdown as FAILED to make it restartable; I did not try to start without it. Only do that when you know the old process is dead. Output: output/09-crash-restart.txt.

## JVM 1 (progress lines, then SIGKILL)
PROGRESS committed-chunks-this-jvm=250 last-ticket=25000
PROGRESS committed-chunks-this-jvm=300 last-ticket=30000
PROGRESS committed-chunks-this-jvm=350 last-ticket=35000
PROGRESS committed-chunks-this-jvm=400 last-ticket=40000
(process killed by SIGKILL; no shutdown hook ran)

## JVM 2
RECOVER: execution 1 is STARTED but its JVM is gone; marking it FAILED so the instance can restart
START run.id=crash-demo tickets=100000 chunk=100
JOB FINISHED: executionId=33 status=COMPLETED seconds=34.2
---- REPORT ----
tickets total=100000 classified=99691 skipped=309 unaccounted=0
step executions:
  {ID=1, JOB=1, STATUS=FAILED, READS=39600, WRITES=39493, SKIPS=107, COMMITS=396, ROLLBACKS=0}
  {ID=33, JOB=33, STATUS=COMPLETED, READS=60400, WRITES=60198, SKIPS=202, COMMITS=604, ROLLBACKS=0}
ledger (committed chunks only): {ROWS_=1000, WRITTEN=99691, SKIPPED=309, ATTEMPTS=103519, R429=1531, U503=458, RETRIES=1989, BILLED=101530, BAD=1839, PT=10809111, CT=1467192, COST=7.605343500}
price table 'EXAMPLE-NOT-A-REAL-PRICE': input 0.50 / output 1.50 per million tokens (configurable placeholder)
provider's own meter (survives kill -9):
  received           104,350
  rejected_429       1,543
  window_429         0
  unavailable_503    462
  billed             102,302
  prompt_tokens      10,891,290
  completion_tokens  1,478,281
billed calls the ledger does not know about (work lost in a crash or rollback): 772
results checksum (sha-256 over ordered rows): 98bdd29feb6d6f8d

The figure restates the transcript: JVM 1’s step execution row says 39,600 reads and 396 commits; JVM 2 read 60,400, and 39,600 + 60,400 is exactly 100,000. Nothing was read twice and nothing was skipped over. The proof is the next transcript, from scripts/compare.sh: the crashed-and-resumed database and the uninterrupted one hold the same results. Output: output/10-compare.txt.

## target/clean/run.log
tickets total=100000 classified=99691 skipped=309 unaccounted=0
ledger (committed chunks only): {ROWS_=1000, WRITTEN=99691, SKIPPED=309, ATTEMPTS=103519, R429=1531, U503=458, RETRIES=1989, BILLED=101530, BAD=1839, PT=10809111, CT=1467192, COST=7.605343500}
billed calls the ledger does not know about (work lost in a crash or rollback): 0
results checksum (sha-256 over ordered rows): 98bdd29feb6d6f8d

## target/crash/jvm2.log
tickets total=100000 classified=99691 skipped=309 unaccounted=0
ledger (committed chunks only): {ROWS_=1000, WRITTEN=99691, SKIPPED=309, ATTEMPTS=103519, R429=1531, U503=458, RETRIES=1989, BILLED=101530, BAD=1839, PT=10809111, CT=1467192, COST=7.605343500}
billed calls the ledger does not know about (work lost in a crash or rollback): 772
results checksum (sha-256 over ordered rows): 98bdd29feb6d6f8d

checksums identical: 98bdd29feb6d6f8d

The line to read is billed calls the ledger does not know about: 772. The provider’s meter, which survived the kill because it lives in a memory-mapped file, says 102,302 billed calls; the ledger says 101,530. The difference is the work of the chunks that were in flight, plus chunks that JVM 1 had logged as committed but the database never kept (explained next). In token terms, from the same two transcripts, that is 10,891,290 − 10,809,111 = 82,179 prompt tokens and 1,478,281 − 1,467,192 = 11,089 completion tokens, about 0.0577 in placeholder units (82,179 × 0.50 + 11,089 × 1.50, per million). Restartability saved 39,600 tickets of re-work; it did not make the crash free, and only a ledger committed with the data can tell you the difference.

A discrepancy I explained but did not isolate. JVM 1 printed committed-chunks-this-jvm=400 before it was killed, yet the database after the crash held 396 committed chunks. My explanation is H2’s default write delay: the MVStore flushes asynchronously (I believe the default is 500 ms, but I could not confirm that against the H2 documentation in this session), so a few “committed” transactions that had not reached the file were lost — atomically, which is why resuming still produced no duplicates and no gaps. I did not re-run with a zero write delay to prove it. The practical point stands for any database: “the process logged commit” and “the data is durable” are two events, and the restart logic only trusts the second.

Smaller traps that cost a compile or a test cycle

• Package moves in Spring Batch 6. ItemProcessor, ItemReader, ItemWriter, Chunk and ExecutionContext now live under org.springframework.batch.infrastructure.item, while jobs, steps and listeners stay under org.springframework.batch.core. The compiler tells you; I list it so a search engine can.
• Jackson 3. Parsing the reply uses tools.jackson.databind.json.JsonMapper, and a node’s text is asString(), not asText().
• Test properties that did nothing. My first multi-scenario tests used SpringApplicationBuilder.properties(...), which sets default properties; application.yml overrides them. The scenarios all ran with 100,000 tickets and failed in confusing ways. Passing the same values as --key=value arguments to run(...) fixed it (SpringContexts.java).
• One JAVA_HOME from the environment. A script that did JAVA_HOME=${JAVA_HOME:-...} kept an older JDK and the compiler reported release version 25 not supported. The scripts now pin JDK 25 unless you override JDK25.

Should you even run this as a Spring Batch job? Often, no. If the work is a few thousand tickets once a quarter, a loop with a rate limiter and a results table is less machinery and perfectly honest. If the provider offers an asynchronous batch endpoint at a lower price (several do; check yours), submit and poll beats a chunk step that holds a transaction open per hundred calls, and you should compare their prices with your numbers before choosing. Spring Batch earns its place when you need what the transcripts above showed: a restart that does not redo or double-bill, a skip table you can act on, and a ledger that matches the data. And be cautious about this article’s numbers: they come from a simulation whose faults are rates I chose. They show how the pieces behave, not how often a real model fails.

Further reading

• Companion code: asmhatre/spring-ai, module batch-llm, with every transcript in output/ and scripts/run-all.sh to regenerate them.
• On this site: Spring Batch on Boot 4.1: Jobs, Steps, Chunk Processing and Restartability; Structured Output in Spring AI 2.0 (the typed-reply alternative to parsing by hand); The LLM Gateway Pattern for Java Microservices (per-provider circuit breakers and caps); Anthropic Claude vs OpenAI vs Gemini in Spring AI 2.0 (retry multiplication across providers).
• Spring Batch reference documentation
• Resilience4j RateLimiter documentation
• Spring AI ChatClient API reference