asmhatreandClaude Sonnet 5.5
6d792f00e0
Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_017ic4b4VQVEJWetTT1dYzyJ
30 lines
2.6 KiB
Markdown
30 lines
2.6 KiB
Markdown
# batch-llm
|
|
|
|
Classify 100,000 synthetic support tickets with a `ChatClient` inside a Spring Batch 6 chunk step: a Resilience4j rate limiter and retry around the model call, skip-and-record for replies that are not valid JSON, restart after `kill -9`, and a cost ledger row committed with every chunk.
|
|
|
|
**No real model was called.** The "provider" is `SimulatedProvider`, a hand-written deterministic `ChatModel` (same idea as the `ScriptedChatModel` in the other modules) that returns valid JSON, malformed JSON, 429s, 503s and slow answers from a seeded fault plan, and enforces its own calls-per-second quota. Tickets are generated in code from their number. Token counts are estimated (characters / 4) and prices in `application.yml` are **placeholders you configure**, not any provider's prices. Nothing here measures a real model's accuracy, latency or price.
|
|
|
|
| | |
|
|
|---|---|
|
|
| Spring Boot | 4.1.2 |
|
|
| Spring Batch | 6.0.6 (managed by Boot; `ChunkOrientedStep`) |
|
|
| Spring AI | 2.0.2 (`spring-ai-client-chat`) |
|
|
| Resilience4j | 2.4.0 (`resilience4j-spring-boot4`) |
|
|
| Java | 25 |
|
|
| Run | `scripts/run-all.sh` (tests, then the 100,000-ticket job twice; `TICKETS=20000 scripts/run-all.sh` is faster) |
|
|
|
|
## Source
|
|
|
|
| File | What it is |
|
|
|---|---|
|
|
| [`TicketClassifier`](src/main/java/com/ankurm/batchllm/TicketClassifier.java) | the `ItemProcessor`: `Retry(RateLimiter(ChatClient call))`, then parse and validate the reply |
|
|
| [`LlmJobConfig`](src/main/java/com/ankurm/batchllm/LlmJobConfig.java) | the `ChunkOrientedStep`: fault tolerant, skip policy, virtual-thread executor, where Spring Batch retry would go |
|
|
| [`ClassificationWriter`](src/main/java/com/ankurm/batchllm/ClassificationWriter.java) | results and the `COST_LEDGER` row in one transaction |
|
|
| [`SkippedTicketListener`](src/main/java/com/ankurm/batchllm/SkippedTicketListener.java) | records skipped tickets in their own transaction |
|
|
| [`SimulatedProvider`](src/main/java/com/ankurm/batchllm/SimulatedProvider.java) | the fake model and its fault plan |
|
|
| [`LlmJobRunner`](src/main/java/com/ankurm/batchllm/LlmJobRunner.java) | start or restart, `JobOperator.recover`, final report |
|
|
|
|
## Output (`output/`)
|
|
|
|
`01-versions` classpath and builder methods, `02-simulated-provider` the fault mix, `03-throttle` with and without a limiter, `04-skip-and-ledger` every ticket accounted for and ledger == provider meter, `05-retry-layers` which layer retries what, `06-skip-limit-and-restart` a model regression stops the job, `08-clean-run` the full 100,000, `09-crash-restart` two JVMs and `kill -9`, `10-compare` the resumed database equals the clean one.
|