LLM Gateway Pattern for Java Microservices with Spring AI 2.0: Routing, Failover and Cost Caps
Build and test an LLM gateway on Spring AI 2.0: hint-based routing, failover that works with the new tool-calling advisor, one circuit breaker per provider, token and dollar caps, and a semantic cache. Every number comes from a run.