Self-correction pass caught this before publishing: the previous commit's test
comment and companion post draft claimed a keepAlive set through
ChatClient.prompt().options(...) never reaches the request Ollama receives,
and worked around it by calling ChatModel.call(Prompt) directly instead. That
claim was never actually verified against /api/ps for the ChatClient path --
only inferred from a failed timing assertion that, it turned out, would have
failed the same way even with a genuinely confirmed unload (see below).
Checked directly: unloading via ChatClient.prompt().options(OllamaChatOptions
.builder()...keepAlive("0")).call() and immediately querying /api/ps shows an
empty model registry, same as the ChatModel path. The options merge works
correctly. Simplified the test back to ChatClient throughout, consistent with
the rest of this series, and removed the incorrect comment.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FtpJvZfg4nvLvtzgJTDWpB
- spring-ai-starter-model-ollama autoconfigures ChatModel/EmbeddingModel from
spring.ai.ollama.* properties alone; no API key anywhere in this module.
- org.testcontainers:ollama and org.testcontainers:junit-jupiter were both
renamed in the Testcontainers 2.x line -- to org.testcontainers:testcontainers-ollama
and org.testcontainers:testcontainers-junit-jupiter respectively -- confirmed by
reading the real testcontainers-bom-2.0.5.pom that Spring Boot 4.1.1 imports
(spring-boot-dependencies -> testcontainers.version=2.0.5). The pre-rename
artifact IDs still exist on Maven Central but are stuck on the 1.x line.
- Unlike every other module in this series, tests drive a real local model
(qwen2.5:0.5b chat, all-minilm embeddings) via a Testcontainers-managed
OllamaContainer started from a baked image (scripts/bake-image.sh), not a
ScriptedChatModel -- the whole point of this post is a real model answering
a real prompt.
- LocalChatAndEmbeddingTest forces a genuine cold state with Ollama's
keep_alive: 0 option (set via ChatModel.call(Prompt) -- ChatClient.options()
does not carry a keepAlive override through to the request in this version)
and confirms the unload actually happened via /api/ps before measuring a
reload, rather than trusting whichever call happens to run first.
- On this quiet sandbox host, even a confirmed-cold reload of the 500MB model
came back in single-digit milliseconds once the underlying image layers were
cached -- eval (generation) time dominates total latency here, not loading.
Captured, not asserted as universal: readers get scripts/bake-image.sh to
get their own numbers.
- Embedding dimension (384, all-minilm) asserted deterministically.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FtpJvZfg4nvLvtzgJTDWpB
ChatClient.CallResponseSpec.entity() mapping LLM JSON to a record (TicketTriage, with a real
enum-constrained Priority field), a List<ActionItem>, and a Map<String,Object> -- every case driven
by a hand-written ScriptedChatModel with no live LLM anywhere.
Key findings, all confirmed by disassembling spring-ai-client-chat-2.0.1.jar and spring-ai-model-2.0.1.jar
rather than trusting docs:
- StructuredOutputValidationAdvisor lives in org.springframework.ai.chat.client.advisor, in the same
spring-ai-client-chat artifact as ToolCallingAdvisor -- unlike the tool-calling module's Tool Search
Advisor pieces, it needs no separate Maven Central artifact or version pin.
- entity(Class, spec -> spec.validateSchema()) is sugar: DefaultCallResponseSpec.resolveAdvisorChain
builds a real StructuredOutputValidationAdvisor from the same JSON schema BeanOutputConverter uses
to parse the response, and pushes it onto the advisor chain for that one call.
- The schema/format instructions are baked into the user message once, up front, by entity() itself,
before the advisor chain runs at all. A validation retry's only contribution is one appended line:
"Output JSON validation failed because of: <the real schema-validator error>" -- each retry
re-augments the ORIGINAL request, not the previous attempt's, so corrections never stack.
- Default maxRepeatAttempts is 3 (4 total attempts); default advisorOrder is 2147481647, near
Ordered.LOWEST_PRECEDENCE.
- Exhausting every retry does NOT throw -- adviseCall's loop just returns the last (still invalid)
response to the caller. Plain entity() with no validation, by contrast, throws immediately on the
same bad JSON, since BeanOutputConverter.convert() is a separate Jackson deserialization step with
no retry loop of its own. Both behaviors are captured from real runs (output/02, output/06).
- Spring AI 2.0's JSON stack is Jackson 3 (tools.jackson.databind), not classic com.fasterxml.jackson --
visible directly in every one of this advisor's constructor and field signatures.
Companion module for "Structured Output in Spring AI 2.0: Records, JSON Schema and Self-Correcting
Responses" on ankurm.com.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FtpJvZfg4nvLvtzgJTDWpB
Every test drives the real Spring AI advisor classes (ToolCallingAdvisor,
ToolSearchToolCallingAdvisor) against a hand-written ScriptedChatModel that queues
real ChatResponse objects instead of calling a live LLM -- confirmed viable because
ChatModel has exactly one abstract method, call(Prompt) (checked with javap).
Covers:
- The plain call/execute/recall loop (WeatherTools, an ordinary @Tool method)
- @Tool(returnDirect = true) skipping the second model round trip entirely
(ServerStatusTools)
- ToolContext: excluded from the model-facing JSON schema (verified against the
real generated schema), still delivered to the tool from caller-supplied data
(UserContextTools)
- A 230-tool synthetic library across six fake domains, generated via
FunctionToolCallback.builder(...) (LargeToolLibrary)
- ToolSearchToolCallingAdvisor + RegexToolIndex: one tool ("toolSearchTool")
offered on the first call instead of 230, with real tool-count and
character-footprint measurements taken off the actual outgoing prompts
Findings recorded in the module's Javadoc rather than silently worked around:
- ToolCallingAdvisor only engages when the request's Prompt carries
ToolCallingChatOptions, built from ChatModel.getOptions().mutate() (not
getDefaultOptions(), a separate default method the request-building path never
calls) -- confirmed by disassembling ToolCallingAdvisor.adviseCall and
DefaultChatClientUtils
- The Tool Search Advisor's own tool is named "toolSearchTool" (camelCase), not
"tool_search_tool" -- confirmed via @Tool(name=...) in the decompiled class
- Its session ID comes from ChatClientRequest.context() (AdvisorSpec.param), not
from ChatClient.toolContext(Map) -- confirmed by disassembling
ToolSearchToolCallingAdvisor.initializeSession
- RegexToolIndex matches on verb/noun substrings, not semantic relevance -- a
real captured search for "look up an invoice" returned 5 lookup_-named tools
across three unrelated domains alongside the one actually wanted
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FtpJvZfg4nvLvtzgJTDWpB
- JWT bearer authentication via spring-boot-starter-oauth2-resource-server, validated against
an RSA keypair DemoJwtIssuer generates and signs with locally, so the whole module runs and
tests deterministically with no external Authorization Server.
- @PreAuthorize on @McpTool methods maps SCOPE_orders:read / SCOPE_orders:write to lookup_order
and refund_order -- confirmed empirically that method security actually applies to a bean the
MCP server autoconfiguration invokes via reflection, since it invokes the Spring-proxied bean.
- SecurityFilterChain requires authentication on every request, so tool discovery (initialize/
tools-list) is rejected before it ever reaches the MCP dispatcher -- no anonymous tool listing.
- ToolAuditAspect logs every tool call through MDC (subject, scopes, tool, outcome), pinned to
@Order(150) -- between AuthorizationInterceptorsOrder.PRE_FILTER (100) and PRE_AUTHORIZE (200)
-- so it wraps @PreAuthorize's interceptor and still logs denied calls, not only successful
ones. Verified with a real Logback ListAppender reading back real MDC contents.
Two real findings worth a note: Spring Boot 4.0 renamed spring-boot-starter-aop to
spring-boot-starter-aspectj (the old artifact stops existing after 4.0.0-M2); and Spring AI's
AbstractSyncMcpToolMethodCallback.createSyncErrorResult concatenates an exception's message with
its root cause's message, which duplicates the text when they're the same exception -- visible
directly in the captured output when @PreAuthorize denies a call ("Access Denied\nAccess Denied").
5/5 tests pass against a real running server over real Streamable HTTP, with real signed JWTs.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FtpJvZfg4nvLvtzgJTDWpB
defaultToolCallbacks(ToolCallbackProvider) wires two real stdio MCP servers (the official
filesystem server and git server) into ChatClient, contrasted with defaultTools(Object) for a
local @Tool method. Every call -- MCP-sourced or local -- is logged through one Micrometer
ObservationHandler<ToolCallingObservationContext>; a first version wired that handler two ways
at once and every call logged twice, which is now a regression test. No real LLM is used
anywhere: every test builds an AssistantMessage.ToolCall by hand and drives it through the real
ToolCallingManager bean against real npx/uvx-launched MCP server processes.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FtpJvZfg4nvLvtzgJTDWpB
Claude