asmhatre/spring-ai
1
0
Fork 0
Code Activity

Add llm-gateway module: routing, failover below the tool-calling advisor, per-provider circuit breakers, dollar caps and token limits, tenant-keyed cache; real OpenAI and Anthropic models against a local fake

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JXVi2GMQ7bR5EmbUFdDj7N
This commit is contained in:
Claude
2026-10-09 10:24:57 +00:00
parent 80cd21f89b
commit cdee85d3f4
51 changed files with 2404 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
llm-gateway/output/04-tool-loop.txt
+19
View File
@@ -0,0 +1,19 @@
# Where failover sits relative to the tool loop
A. failover below the advisor (this module)
tool executions: 1
trail: [openai: ok, openai: failed with status 503, claude: ok]
answer: Refunded as refund-A17-1 (from claude)
what claude was sent: UserMessage, AssistantMessage, ToolResponseMessage
B. retry above the advisor (one ChatClient per provider)
tool executions: 2
answer: Refunded as refund-A17-2
C. usage across a two-call tool loop (calls reported 120+15 and 160+12)
usage on the final ChatResponse: prompt=280 completion=27
usage summed per model call: prompt=280 completion=27
D. the same loop with the first call on a cheaper provider (0.75/3.75 then 2.00/10.00)
priced per model call: 587 microdollars
total usage at the last provider's price: 830 microdollars