asmhatre/spring-ai
1
0
Fork 0
Code Activity
Files
6d792f00e0e937fa6c1f2c727b9f5aa9d0c729f5
spring-ai/batch-llm
T
History
asmhatreandClaude Sonnet 5.5 6d792f00e0 Add batch-llm module: 100,000 synthetic tickets classified by a ChatClient in a Spring Batch 6 chunk step with Resilience4j rate limiting and retry, skip-and-record for malformed JSON, restart after kill -9 in a second JVM, and a per-chunk cost ledger. The model is a deterministic simulated ChatModel; prices are configurable placeholders. No provider called. 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017ic4b4VQVEJWetTT1dYzyJ
2026-10-29 06:10:22 +00:00
..
output
Add batch-llm module: 100,000 synthetic tickets classified by a ChatClient in a Spring Batch 6 chunk step with Resilience4j rate limiting and retry, skip-and-record for malformed JSON, restart after kill -9 in a second JVM, and a per-chunk cost ledger. The model is a deterministic simulated ChatModel; prices are configurable placeholders. No provider called.
2026-10-29 06:10:22 +00:00
scripts
Add batch-llm module: 100,000 synthetic tickets classified by a ChatClient in a Spring Batch 6 chunk step with Resilience4j rate limiting and retry, skip-and-record for malformed JSON, restart after kill -9 in a second JVM, and a per-chunk cost ledger. The model is a deterministic simulated ChatModel; prices are configurable placeholders. No provider called.
2026-10-29 06:10:22 +00:00
src
Add batch-llm module: 100,000 synthetic tickets classified by a ChatClient in a Spring Batch 6 chunk step with Resilience4j rate limiting and retry, skip-and-record for malformed JSON, restart after kill -9 in a second JVM, and a per-chunk cost ledger. The model is a deterministic simulated ChatModel; prices are configurable placeholders. No provider called.
2026-10-29 06:10:22 +00:00
.gitignore
Add batch-llm module: 100,000 synthetic tickets classified by a ChatClient in a Spring Batch 6 chunk step with Resilience4j rate limiting and retry, skip-and-record for malformed JSON, restart after kill -9 in a second JVM, and a per-chunk cost ledger. The model is a deterministic simulated ChatModel; prices are configurable placeholders. No provider called.
2026-10-29 06:10:22 +00:00
pom.xml
Add batch-llm module: 100,000 synthetic tickets classified by a ChatClient in a Spring Batch 6 chunk step with Resilience4j rate limiting and retry, skip-and-record for malformed JSON, restart after kill -9 in a second JVM, and a per-chunk cost ledger. The model is a deterministic simulated ChatModel; prices are configurable placeholders. No provider called.
2026-10-29 06:10:22 +00:00
README.md
Add batch-llm module: 100,000 synthetic tickets classified by a ChatClient in a Spring Batch 6 chunk step with Resilience4j rate limiting and retry, skip-and-record for malformed JSON, restart after kill -9 in a second JVM, and a per-chunk cost ledger. The model is a deterministic simulated ChatModel; prices are configurable placeholders. No provider called.
2026-10-29 06:10:22 +00:00

README.md

batch-llm

Classify 100,000 synthetic support tickets with a ChatClient inside a Spring Batch 6 chunk step: a Resilience4j rate limiter and retry around the model call, skip-and-record for replies that are not valid JSON, restart after kill -9, and a cost ledger row committed with every chunk.

No real model was called. The "provider" is SimulatedProvider, a hand-written deterministic ChatModel (same idea as the ScriptedChatModel in the other modules) that returns valid JSON, malformed JSON, 429s, 503s and slow answers from a seeded fault plan, and enforces its own calls-per-second quota. Tickets are generated in code from their number. Token counts are estimated (characters / 4) and prices in application.yml are placeholders you configure, not any provider's prices. Nothing here measures a real model's accuracy, latency or price.
Spring Boot 4.1.2
Spring Batch 6.0.6 (managed by Boot; ChunkOrientedStep)
Spring AI 2.0.2 (spring-ai-client-chat)
Resilience4j 2.4.0 (resilience4j-spring-boot4)
Java 25
Run scripts/run-all.sh (tests, then the 100,000-ticket job twice; TICKETS=20000 scripts/run-all.sh is faster)

Source

File What it is
TicketClassifier the ItemProcessor: Retry(RateLimiter(ChatClient call)), then parse and validate the reply
LlmJobConfig the ChunkOrientedStep: fault tolerant, skip policy, virtual-thread executor, where Spring Batch retry would go
ClassificationWriter results and the COST_LEDGER row in one transaction
SkippedTicketListener records skipped tickets in their own transaction
SimulatedProvider the fake model and its fault plan
LlmJobRunner start or restart, JobOperator.recover, final report

Output (output/)

01-versions classpath and builder methods, 02-simulated-provider the fault mix, 03-throttle with and without a limiter, 04-skip-and-ledger every ticket accounted for and ledger == provider meter, 05-retry-layers which layer retries what, 06-skip-limit-and-restart a model regression stops the job, 08-clean-run the full 100,000, 09-crash-restart two JVMs and kill -9, 10-compare the resumed database equals the clean one.

View Git Blame Copy Permalink