returnDirect, where the tool’s own answer skips that paraphrase entirely; ToolContext, for data a tool needs that the model must never see or supply; and Tool Search Advisor, which stops a 230-tool library from putting all 230 definitions in front of the model on every single call.
Nothing below calls a real LLM. Every test in this article’s companion repo drives the real Spring AI advisor classes — the same ToolCallingAdvisor and ToolSearchToolCallingAdvisor a production app would use — against a hand-written ChatModel that returns a queued, pre-programmed response instead of calling an API. That sounds like it should be a weaker kind of proof. It’s the opposite: it means every number and every tool count in this article was read directly off the real advisor’s real behavior, not off documentation describing what it’s supposed to do.
Versions. Spring Boot 4.1.1 and Spring AI 2.0.1, on Java 25 (LTS) — the same baseline as the rest of this series. The Tool Search Advisor pieces (ToolSearchToolCallingAdvisor,RegexToolIndex) live in two separately versioned Maven Central artifacts,spring-ai-tool-search-advisorandspring-ai-tool-search-tool, both at 2.0.1 — confirmed against Maven Central’s own metadata, since neither is covered byspring-ai-bomand both need pinning explicitly inpom.xml.
The plain case: a tool, a round trip, an answer
The simplest possible tool is an ordinary method with an@Tool annotation and nothing else:
@Tool(name = "current_weather", description = "Get the current weather conditions for a named city.")
public WeatherReport currentWeather(@ToolParam(description = "The city name, e.g. Boston") String city) {
return FIXTURE_DATA.get(city);
}ChatClient wired with this tool “what’s the weather in Boston?” and two separate calls happen to the model, not one:
model call count: 2
call 2 -- advisor sends the tool result back to the model:
tool response: ToolResponse[id=call-1, name=current_weather, responseData={"city":"Boston","temperatureCelsius":14.5,"conditions":"Overcast","humidityPercent":71}]
final answer returned to the caller:
It's 14.5C and overcast in Boston right now, with 71% humidity.ToolCallingAdvisor, registered on the ChatClient like any other advisor:
ChatClient.builder(chatModel)
.defaultToolCallbacks(toolLibrary)
.defaultAdvisors(ToolCallingAdvisor.builder().build())
.build();Going deeper on this section
- Companion repo: PlainToolCallingRoundTripTest.java
- Related: Build an MCP Server with Spring AI 2.0 — the
@McpToolshape of the same idea, exposed over MCP instead of called locally
What actually changed from Spring AI 1.x
In Spring AI 1.x, the call/execute/recall loop above lived inside eachChatModel implementation — OpenAiChatModel, AnthropicChatModel, and every other provider each had their own internal copy of essentially the same logic, invoking a ToolCallingManager from inside their own call()/stream() methods. Every provider integration had to get that loop right on its own, and a bug fix or a new capability in the loop meant touching every provider.
In Spring AI 2.0, that logic moved out of every ChatModel and into ToolCallingAdvisor, which is confirmed to live entirely at the ChatClient advisor layer — the class implements CallAdvisor, StreamAdvisor, and ToolAdvisor, and its logic never touches a provider-specific type. One implementation drives the loop for every model.
ToolCallingAdvisor only depends on the plain ChatModel interface, and ChatModel has exactly one abstract method (call(Prompt) — confirmed with javap; every other method on the interface has a default implementation), a hand-written test double that queues canned responses is a legitimate, fully-real ChatModel from the advisor’s point of view, not a mock standing in for one.
public class ScriptedChatModel implements ChatModel {
private final Queue<ChatResponse> script;
@Override
public ChatResponse call(Prompt prompt) {
capturedPrompts.add(prompt);
return script.poll();
}
}Going deeper: the one line that makes ScriptedChatModel actually work
ToolCallingAdvisor.adviseCall‘s very first check, confirmed by disassembling the class, is an instanceof ToolCallingChatOptions test on the outgoing request’s options — if it fails, the advisor passes the call straight through with no tool loop at all. That options object is built from chatModel.getOptions().mutate(), a separate default method from getDefaultOptions() that a first version of this repository’s ScriptedChatModel overrode instead, by mistake, and watched every test get back an empty answer after exactly one model call. Real providers return their own ToolCallingChatOptions implementation from getOptions(); ScriptedChatModel has to do the same thing explicitly.
Going deeper on this section
- Companion repo: ChatClientFactory.java, whose Javadoc walks through the
getOptions()finding above in full - Official reference: Spring AI – Tool Calling
returnDirect: when the tool’s answer is the answer
Every round trip in the previous section exists so the model can turn a tool’s raw JSON into a sentence. Sometimes that’s wasted work — a status dashboard, a generated report, a confirmation payload is already the answer, and asking the model to paraphrase it just adds latency and a chance it states one of the numbers wrong.@Tool(returnDirect = true) tells the advisor to skip the second call entirely and hand the tool’s own return value straight back:
@Tool(name = "server_status", description = "Get the live status of an internal service.", returnDirect = true)
public ServerStatus serverStatus(@ToolParam(description = "The service name, e.g. orders-api") String serviceName) {
return new ServerStatus(serviceName, "UP", ACTIVE_CONNECTIONS.get(serviceName), Instant.now(clock));
}ScriptedChatModel would throw on an empty queue and the test would fail loudly, not pass by accident.
model call count: 1
raw content returned to the caller (the tool's own JSON, unparaphrased):
{"service":"orders-api","status":"UP","activeConnections":214,"checkedAt":"2026-09-23T18:00:00Z"}returnDirect is a statement that this particular tool’s output is fit to hand to whatever’s downstream of the model as-is, which is true for a dashboard payload and not true for most conversational answers.
returnDirect is a data-shape decision, not just a performance one. Reach for it when the tool’s return type is already what the caller wants to consume — a UI component render, a file, a structured payload another service parses. Reach for it less when a human is going to read the answer as prose; a model’s paraphrase is often exactly the value-add over the raw data.
Going deeper on this section
- Companion repo: ReturnDirectSkipsSecondRoundTest.java
ToolContext: data the model must never see or choose
Some data a tool needs shouldn’t come from the model at all — the authenticated caller’s user ID, a tenant, a feature flag. Putting it in an ordinary@ToolParam means trusting the model to supply it correctly, which also means trusting it not to supply someone else’s. ToolContext is Spring AI’s way around that: a parameter the method declares by type, filled in by the caller’s own code, and never exposed to the model at all.
@Tool(name = "my_account", description = "Get the current user's own account summary.")
public AccountSummary myAccount(ToolContext toolContext) {
Object userId = toolContext.getContext().get("userId");
return ACCOUNTS.get(userId.toString());
}my_account input schema sent to the model:
{
"$schema" : "https://json-schema.org/draft/2020-12/schema",
"type" : "object",
"properties" : { },
"required" : [ ],
"additionalProperties" : false
}userId property anywhere — the schema is recognized by type, not skipped by a naming convention. The model’s actual tool call arrives with empty arguments, and the tool still produces the right answer, because the caller supplied the real value out-of-band:
client.prompt()
.user("What's my account tier?")
.toolContext(Map.of("userId", "alice"))
.call()
.content();UserContextTools trusts whatever’s in that map outright, which is fine for a demo and wrong for production — a real deployment populates ToolContext from the authenticated principal on the server side, the same SecurityContextHolder pattern this series’ MCP security article uses, never from anything a client sends.
Going deeper on this section
- Companion repo: UserContextTools.java
- Related: Securing an MCP Server with Spring Security 7 — where a real authenticated principal, not a hardcoded map, is the source of this kind of data
A 230-tool library, and what it costs to just register them all
Every tool this far has been one of a handful. Real internal platforms don’t stay that way — an HR system, a finance system, a deploy pipeline, a CRM, a logistics system, and a security console each contribute a dozen or two tools, and the total climbs past a hundred fast. This repository generates 230 of them programmatically, across six fake business domains, withFunctionToolCallback.builder(name, Function) instead of 230 hand-written @Service classes:
FunctionToolCallback.builder(name, (SyntheticToolRequest request) -> verb + " " + nounPhrase + " ok")
.description(description)
.inputType(SyntheticToolRequest.class)
.build();total tools: 230
crm 40 tools
devops 38 tools
finance 40 tools
hr 36 tools
logistics 40 tools
security 36 toolsChatClient with the plain ToolCallingAdvisor from earlier, and every one of those 230 names, descriptions, and JSON schemas goes into the model’s context on every single call — whether the question needs one of them or none of them. Measuring the actual text, not guessing at it:
total characters of name + description + input schema, ALL 230 tools: 69958
characters of name + description + input schema, toolSearchTool ONLY: 463
reduction on the first call of a conversation: 99.3%Going deeper on this section
- Companion repo: ToolLibrarySizeTest.java and ToolDescriptionFootprintTest.java
Tool Search Advisor: search first, call second
ToolSearchToolCallingAdvisor is the fix: instead of putting every registered tool’s definition in front of the model, it offers exactly one — a toolSearchTool that searches a ToolIndex built from the real registered library — plus a system-message instruction telling the model to search before it guesses.
ToolSearchToolCallingAdvisor.builder()
.toolIndex(new RegexToolIndex())
.maxResults(5)
.sessionIdKeyName(ToolSearchTool.TOOL_SEARCH_TOOL_SESSION_ID_KEY)
.systemMessageSuffix("Use the toolSearchTool to find the specific tool you need before calling it. "
+ "Do not guess a tool name that was not returned by a search.")
.build();RegexToolIndex is the deterministic, no-embedding-model implementation of ToolIndex — the other two real implementations found in the dependency, LuceneToolIndex and VectorToolIndex, need a search engine or an embedding model respectively; this article picks the one with nothing else to stand up.
ToolCallingChatOptions.getToolCallbacks() on the real outgoing prompt, not off a log line describing what the advisor is supposed to have done:
tools registered on the ChatClient: 230
tools OFFERED to the model on call 1 (before any search): 1
[toolSearchTool]
tools OFFERED to the model on call 2 (after it searched for "look up an invoice"): 6
[crm_lookup_support_ticket, finance_lookup_invoice, hr_lookup_employee_record, hr_lookup_open_requisition, hr_lookup_pto_balance, toolSearchTool]
final answer: Invoice INV-1001 was found in the finance system.finance_lookup_invoice correctly. The session ID keying that search index into the advisor’s per-conversation cache doesn’t come from ChatClient.toolContext(Map) the way UserContextTools‘s data did two sections ago — it comes from a different mechanism entirely:
client.prompt()
.user("Can you look up invoice INV-1001 for me?")
.advisors(a -> a.param(ToolSearchTool.TOOL_SEARCH_TOOL_SESSION_ID_KEY, sessionId))
.call()
.content();ToolSearchToolCallingAdvisor.initializeSession shows exactly why: it reads the session ID off ChatClientRequest.context() — the advisor-param map that AdvisorSpec.param(...) populates — not off ToolCallingChatOptions.getToolContext(). The two context maps look interchangeable from the outside and are not.
Going deeper: why 6 tools came back, not 1
RegexToolIndex is honestly named — it converts the search query into a regular expression and matches it against tool names, descriptions, and parameter text, with no embedding model and no notion of semantic closeness. Searching “look up an invoice” matched every tool whose name or description contains “lookup” or “invoice”-adjacent terms, which is why hr_lookup_pto_balance and crm_lookup_support_ticket came back alongside the actually-relevant finance_lookup_invoice. That’s not a bug in this article’s setup; it’s the real, honest behavior of the deterministic index this article deliberately chose over VectorToolIndex, which would narrow this list using an embedding model instead of a verb-matching regex.
Going deeper on this section
- Companion repo: ToolSearchProgressiveDisclosureTest.java
- Origin of this feature: Smart Tool Selection: Achieving 34-64% Token Savings with Spring AI’s Dynamic Tool Discovery
Should you reach for progressive disclosure at all?
Not for the handful of tools most applications actually register. The plainToolCallingAdvisor from the start of this article is simpler, has one fewer moving part, and costs nothing extra when a library is small enough that offering all of it fits comfortably in context — which is most applications, most of the time. Tool Search Advisor earns its complexity specifically at the scale this article’s synthetic library sits at: dozens of tools across several unrelated domains, where most of a given conversation’s tools are irrelevant to it.
The regex index is a starting point, not a destination. A tool library that grows past what verb-and-noun regex matching can tell apart on its own — several tools with genuinely similar names doing genuinely different things — is the point at whichVectorToolIndexand an embedding model earn their keep overRegexToolIndex‘s determinism. Reach for the deterministic index while you can reason about exactly why a search matched; reach for the vector index once you can’t.
Every Spring AI article on this site
| Article | Covers |
|---|---|
| Spring AI 2.0 in 10 Minutes: ChatClient on Spring Boot 4.1 | the beginner-level ChatClient build this article’s versions and setup follow |
| Build an MCP Server with Spring AI 2.0 | exposing tools over MCP instead of calling them locally through a ChatClient |
| Spring AI MCP Client: Calling External MCP Servers from ChatClient | tools that live in a separate process, registered the same way as a local @Tool method |
| Securing an MCP Server with Spring Security 7 | OAuth2 scopes per tool — where ToolContext‘s real userId should come from in production |
| Spring AI 1.x to 2.0: The Migration Guide | what breaks, and what breaks silently, upgrading an existing 1.x application |
| Production-Grade RAG with Spring AI | chunking, ingestion, retrieval, reranking, and a faithfulness check against pgvector |
| Spring AI RAG in Java: Complete Code Tour | the same RAG project, file by file |
Further reading
- Companion repository for this article: asmhatre/spring-ai, tool-calling module
- Origin of Tool Search Advisor: Smart Tool Selection: Achieving 34-64% Token Savings with Spring AI’s Dynamic Tool Discovery
- Official reference: Spring AI – Tool Calling
- Official reference: Spring AI – Chat Client API
No Comments yet!