asmhatre/spring-ai
1
0
Fork 0
Code Activity

Add batch-llm module: 100,000 synthetic tickets classified by a ChatClient in a Spring Batch 6 chunk step with Resilience4j rate limiting and retry, skip-and-record for malformed JSON, restart after kill -9 in a second JVM, and a per-chunk cost ledger. The model is a deterministic simulated ChatModel; prices are configurable placeholders. No provider called.

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017ic4b4VQVEJWetTT1dYzyJ
This commit is contained in:
asmhatre
2026-10-29 06:10:22 +00:00
co-authored by Claude Sonnet 5.5
parent aa8b1604ed
commit 6d792f00e0
48 changed files with 2093 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
batch-llm/README.md
+29
View File
@@ -0,0 +1,29 @@
# batch-llm
Classify 100,000 synthetic support tickets with a `ChatClient` inside a Spring Batch 6 chunk step: a Resilience4j rate limiter and retry around the model call, skip-and-record for replies that are not valid JSON, restart after `kill -9`, and a cost ledger row committed with every chunk.
**No real model was called.** The "provider" is `SimulatedProvider`, a hand-written deterministic `ChatModel` (same idea as the `ScriptedChatModel` in the other modules) that returns valid JSON, malformed JSON, 429s, 503s and slow answers from a seeded fault plan, and enforces its own calls-per-second quota. Tickets are generated in code from their number. Token counts are estimated (characters / 4) and prices in `application.yml` are **placeholders you configure**, not any provider's prices. Nothing here measures a real model's accuracy, latency or price.
| | |
|---|---|
| Spring Boot | 4.1.2 |
| Spring Batch | 6.0.6 (managed by Boot; `ChunkOrientedStep`) |
| Spring AI | 2.0.2 (`spring-ai-client-chat`) |
| Resilience4j | 2.4.0 (`resilience4j-spring-boot4`) |
| Java | 25 |
| Run | `scripts/run-all.sh` (tests, then the 100,000-ticket job twice; `TICKETS=20000 scripts/run-all.sh` is faster) |
## Source
| File | What it is |
|---|---|
| [`TicketClassifier`](src/main/java/com/ankurm/batchllm/TicketClassifier.java) | the `ItemProcessor`: `Retry(RateLimiter(ChatClient call))`, then parse and validate the reply |
| [`LlmJobConfig`](src/main/java/com/ankurm/batchllm/LlmJobConfig.java) | the `ChunkOrientedStep`: fault tolerant, skip policy, virtual-thread executor, where Spring Batch retry would go |
| [`ClassificationWriter`](src/main/java/com/ankurm/batchllm/ClassificationWriter.java) | results and the `COST_LEDGER` row in one transaction |
| [`SkippedTicketListener`](src/main/java/com/ankurm/batchllm/SkippedTicketListener.java) | records skipped tickets in their own transaction |
| [`SimulatedProvider`](src/main/java/com/ankurm/batchllm/SimulatedProvider.java) | the fake model and its fault plan |
| [`LlmJobRunner`](src/main/java/com/ankurm/batchllm/LlmJobRunner.java) | start or restart, `JobOperator.recover`, final report |
## Output (`output/`)
`01-versions` classpath and builder methods, `02-simulated-provider` the fault mix, `03-throttle` with and without a limiter, `04-skip-and-ledger` every ticket accounted for and ledger == provider meter, `05-retry-layers` which layer retries what, `06-skip-limit-and-restart` a model regression stops the job, `08-clean-run` the full 100,000, `09-crash-restart` two JVMs and `kill -9`, `10-compare` the resumed database equals the clean one.