ChatClient call, sees the request on the way to the model and the answer on the way back, and can change, log, count or refuse either. Spring AI already uses advisors for chat memory and for tool calling, so you are using them whether you know it or not.
This article writes three of your own — a logger, a PII redactor and a token budget — and spends most of its length on the parts that are easy to get wrong: what “order” means when you have several advisors, why streaming needs different code from a plain call, and where the redactor has to sit if you want the log and the chat memory to stay clean. There is no live model in any of it. A recording stub that notes exactly what it was sent is what an advisor test needs, so every claim below comes from a test you can run with no API key. Depth is in expandable sections; read straight through or open only what you need.
Versions this was written and run against. Spring Boot 4.1.1, Spring AI 2.0.1 (spring-ai-client-chat2.0.1), Reactor Core 3.8.7, JTokkit 1.1.0 (the token estimator) and Java 25. All the code is in theadvisorsmodule of asmhatre/spring-ai; every console block below is quoted from a file under itsoutput/directory, written by a test that asserts the same facts. Two consecutive full runs produce byte-identical files.
An advisor is a function wrapped around the model call
Think of aChatClient call as a request that has to travel to the model and an answer that has to travel back. An advisor stands on that road. Each advisor receives the request, may change it, hands it to the next one, and when the answer returns it gets a second look on the way out. With several advisors they nest like layers of an onion, and the model is at the centre.
CallAdvisor, whose single method receives the request and the rest of the chain. Calling chain.nextCall(request) is “pass it on”; everything before that line happens on the way in, everything after it on the way out (LoggingAdvisor.java):
public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain) {
long start = nanoClock.getAsLong();
sink.accept("[" + name + "] request " + describe(request));
try {
ChatClientResponse response = chain.nextCall(request);
sink.accept("[" + name + "] response " + describe(response) + " took " + millis(start) + " ms");
return response;
}
catch (RuntimeException e) {
sink.accept("[" + name + "] failed " + e.getClass().getSimpleName() + " after " + millis(start) + " ms");
throw e;
}
}System.nanoTime).
A call with content logging off, which is the default (from output/05-logging-advisor.txt):
call, content logging off (the default):
[LoggingAdvisor] request messages=2 roles=SU
[LoggingAdvisor] response chars=14 tokens=12+3 took 5 msoutput/06-logging-failure.txt):
caller got: IllegalStateException: provider returned 503
[LoggingAdvisor] request messages=1 roles=U
[LoggingAdvisor] failed IllegalStateException after 5 msGoing deeper: what a ChatClientRequest and ChatClientResponse are
Both are immutable records. A request carries the
Prompt (the list of messages plus options) and a context map; a response carries the ChatResponse from the model and its own context. Nothing is changed in place: an advisor that wants to change the prompt builds a copy with request.mutate() and passes the copy on. That is why Texts.java exists, and it is how the redactor works below. The context map is how advisors talk to each other and to the call site: a value set with .advisors(a -> a.param(key, value)) arrives in request.context(), and a copy made with request.mutate().context(key, value) travels to every advisor downstream. The context map on an incoming request turned out to be modifiable, so nothing stops you writing into it directly — do not; copy it (output/17-context.txt).
ContextAndImmutabilityTest.java— request context travels down and response context travels up.- Spring AI reference: Advisors API.
Order is a number, not a position in your list
Every advisor has agetOrder(), an integer. Spring AI sorts the chain by it: the lower number is the outer layer, it sees the request first and the answer last. The order in which you list the advisors when you build the client does not decide anything except ties. Spring’s own advisors use numbers counted from Ordered.HIGHEST_PRECEDENCE (which is Integer.MIN_VALUE), so “HIGHEST + 100” is a small step inside the very outermost position.
This module writes its three numbers down in one place, so the reasoning lives next to the constants (Orders.java):
public static final int PII_REDACTION = Ordered.HIGHEST_PRECEDENCE + 100;
public static final int TOKEN_BUDGET = Ordered.HIGHEST_PRECEDENCE + 250;
public static final int LOGGING = Ordered.HIGHEST_PRECEDENCE + 400;output/01-chain-order.txt):
registered as C300, A100, B200 (the number is HIGHEST_PRECEDENCE + n)
call: A100> B200> C300> model C300< B200< A100<
stream: A100> B200> C300> model C300< B200< A100<
plus one advisor added on the request with .advisors(...) at n=150:
call: A100> R150> B200> C300> model C300< B200< R150< A100<output/20-built-in-orders.txt):
SimpleLoggerAdvisor getOrder() = 0
SafeGuardAdvisor getOrder() = 0
MessageChatMemoryAdvisor getOrder() = -2147483448 (HIGHEST_PRECEDENCE + 200)
Tool Calling Advisor getOrder() = -2147483348 (HIGHEST_PRECEDENCE + 300)
Advisor.DEFAULT_CHAT_MEMORY_PRECEDENCE_ORDER = HIGHEST_PRECEDENCE + 200SimpleLoggerAdvisor and SafeGuardAdvisor both report order 0 — far inside everything numbered from HIGHEST_PRECEDENCE, right next to the model. Two more facts about the chain surprise people. First, a ChatClient adds advisors you never registered.
One custom advisor at +100 (the test calls it Dump), and the chain a call actually runs through (from output/03-chain-contents.txt):
one custom advisor at HIGHEST_PRECEDENCE + 100; the chain a call runs through:
Dump HIGHEST_PRECEDENCE + 100
Tool Calling Advisor HIGHEST_PRECEDENCE + 300
call LOWEST_PRECEDENCEToolCallingAdvisor replaces the default rather than adding a second one (the chain still lists one). Second, and worse, two advisors with the same number.
Two advisors given the same number, registered in both orders (from output/02-chain-order-ties.txt):
registered X, Y -> Y> X> model X< Y<
registered Y, X -> X> Y> model Y< X<
For equal numbers the advisor registered LAST runs first (outermost).ChainOrderTest.java prints it rather than assuming it). The fix is never to rely on it: give every advisor of yours a distinct number, and keep them in a constants class like Orders.
Going deeper: what a lower or higher number does not mean
The number is a sort key, not a priority in the sense of “more important”. Order also does not depend on which advisors are defaults on the client and which are added per request with
.advisors(...): the request-level probe in output/01-chain-order.txt slots in by number like the others. And the numbers are not a stable public contract across versions. The chat-memory advisor was at HIGHEST_PRECEDENCE + 1000 in Spring AI 1.1.8 and is at +200 in 2.0.1, so an advisor you numbered to sit “just outside memory” on 1.x can silently end up on the other side after an upgrade (the chat memory article has the comparison). Read the built-in numbers with a test like the one above, instead of trusting a table in a blog post, this one included.
BuiltInOrdersTest.java— prints the numbers of the advisors that ship with Spring AI.ChainOrderTest.java— the ordering, tie and chain-contents tests.- Spring AI 1.x to 2.0 migration guide.
A stream is not a call: BaseAdvisor and the last-chunk trap
So far the model answered in one piece. .stream() answers in many small pieces, called chunks, as the model produces them, so a chat window can show words appearing. That changes what an advisor can do, and it needs a different method: StreamAdvisor.adviseStream(...) returns a Flux of chunks (a Reactor stream) instead of one response. An advisor that should work for both calls and streams implements both interfaces, as all three of ours do.
Spring AI also ships BaseAdvisor, a convenience: you write before(request) and after(response) and it implements both interfaces for you. It is the natural first thing to reach for, and on a call it does exactly what you expect. On a stream it has a trap, which the test below measures with a five-chunk answer.
The same advisor on a call and on a stream (from output/04-base-advisor-stream.txt):
call: before x1, after x1
before ran on thread: the caller's thread
stream: the model streamed 5 chunks: one |two |thre|e fo|ur
before x1, after x1
after saw only: "ur"
before ran on thread: a boundedElastic worker, not the caller's thread
stream whose last chunk has no finish reason: before x1, after x0after ran once, and what it received was the last chunk only — ur — not the answer. An after that redacts, counts or restores text would silently process a fraction of the reply. The same transcript has a second, quieter finding: when the last chunk carries no finish reason, after did not run at all (“after x0”), which is what you get if the provider’s final chunk arrives without a finish reason. And before ran on a different thread for the stream than for the call, because it is scheduled on a Reactor worker; anything you keep in a thread-local will not be there.
The rule this produces. UseThe logger shows the direct approach working. On a stream it counts chunks and characters as they pass, and writes one summary when the stream completes, so it never needs the whole text at once (BaseAdvisorfor work that only needs the request (adding a system message, tagging the context) or that only needs the final chunk. For anything that must see every chunk, implementCallAdvisorandStreamAdvisordirectly. All three advisors in this article do, for that reason.
output/05-logging-advisor.txt).
The same logger on a three-chunk stream (from output/05-logging-advisor.txt):
stream:
[LoggingAdvisor] request messages=2 roles=SU
[LoggingAdvisor] complete 3 chunks, 14 chars tokens=12+3 took 5 msGoing deeper: why the stream branch uses Flux.defer
Each stream advisor in this module does its setup inside
Flux.defer(() -> ...). A Flux returned from adviseStream is a recipe, not a running thing: nothing happens until someone subscribes. Work done outside the defer runs at the moment the advisor is called, which is not necessarily when a subscriber starts the stream, and a stream subscribed twice would share it; timers, counters and the per-request state of the logger and the budget belong inside it, so each subscription starts fresh. The streaming behaviour of BaseAdvisor above was measured, not read: BaseAdvisorStreamTest.java.
LoggingAdvisor.java— the stream branch is the second method.- Project Reactor reference for
Flux.deferand operators used here.
Redact PII before the model sees it, and put it back afterwards
“PII” is personally identifiable information: an email address, a phone number, a card number. The rule for chat models is simple to state. Whatever you send to a hosted model leaves your systems, so anything the model does not need to see should not be in the request. It usually does not need the address itself; it needs to know that an address is there. So the advisor swaps each sensitive value for a numbered placeholder such as<EMAIL_1> on the way in, and swaps the placeholders back for the real values in the answer on the way out, so the user still reads “I will write to [email protected]” and never sees a placeholder.
The advisor is short because the two halves are lambdas over a per-conversation Session that remembers which value became which placeholder. Texts.mapRequestText builds the rewritten copy of the request (PiiRedactionAdvisor.java):
public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain) {
PiiRedactor.Session session = sessionFor(request);
ChatClientResponse response = chain.nextCall(Texts.mapRequestText(request, session::redact));
return restore ? Texts.mapResponseText(response, session::restore) : response;
}output/07-pii-redaction.txt):
caller sends: Hi, I'm Priya. Email [email protected] or call +91 98765 43210. Card 4111 1111 1111 1111, order 1234 5678 9012 3456, and again [email protected].
model was sent: Hi, I'm Priya. Email <EMAIL_1> or call <PHONE_1>. Card <CARD_1>, order 1234 5678 9012 3456, and again <EMAIL_1>.
caller receives: You said: Hi, I'm Priya. Email [email protected] or call +91 98765 43210. Card 4111 1111 1111 1111, order 1234 5678 9012 3456, and again [email protected].
with restore switched off, the caller receives:
You said: Hi, I'm Priya. Email <EMAIL_1> or call <PHONE_1>. Card <CARD_1>, order 1234 5678 9012 3456, and again <EMAIL_1>.restore switched off, the last line shows the caller receiving the placeholders, which is the right setting when the answer is going to a log rather than to a person.
Pattern matching is a floor, not compliance. The redactor finds what looks like an email, a Luhn-valid card number or a phone number. It does not find a person’s name, a street address, a passport or tax number in a shape it does not know, an email written as “priya (at) example (dot) com”, or a card number with a typo. The test prints exactly that, unchanged, so you can see the gap for yourself (next block, from output/08-pii-limits.txt). If a law or a contract says certain data must not leave, a regular expression is not the control; a named-entity model, a data-loss-prevention service or not sending the field at all is.
in: My name is Priya Sharma and I live at 14 Hill Road, Bandra, Mumbai 400050.
out: My name is Priya Sharma and I live at 14 Hill Road, Bandra, Mumbai 400050.
in: Passport N1234567, PAN ABCDE1234F.
out: Passport N1234567, PAN ABCDE1234F.
in: Write to priya (at) example (dot) com
out: Write to priya (at) example (dot) comGoing deeper: rewriting a request without breaking it
Texts.java rewrites the text of every system, user and assistant message with mutate() and leaves everything else in the request alone — options, metadata, media. One deliberate exception: an assistant message that carries tool calls is passed through untouched, because rewriting it can change the arguments the model asked a tool to run. The response side uses ChatResponse.builder().from(original), so metadata such as token usage is carried over to the rewritten response instead of being dropped. The patterns themselves are in PiiRedactor.java.
PiiRedactionTest.java— the tests behind transcripts 07 to 09.- OWASP Top 10 for LLM Applications — sensitive information disclosure is on the list.
Streaming splits your placeholders in half
The restore step is easy on a call, where the whole answer arrives at once. On a stream it is where the redactor breaks. The model produces text in chunks of whatever size it likes, and nothing makes it respect the boundaries of your placeholder. A<EMAIL_1> can arrive as <EM, then AIL_1, then >, and a restore that looks at one chunk at a time never sees the whole placeholder, so it restores nothing. The test forces that by having the stub stream five-character chunks.
The naive version and the real one, on the same stream (from output/09-pii-stream-boundary.txt):
the model streams 5-character chunks: Sure,| I wi|ll wr|ite t|o <EM|AIL_1|> now|.
restore each chunk on its own:
chunks: Sure,| I wi|ll wr|ite t|o <EM|AIL_1|> now|.
joined: Sure, I will write to <EMAIL_1> now.
PiiRedactionAdvisor (holds back from an unfinished "<"):
chunks: Sure,| I wi|ll wr|ite t|o |[email protected] now|.
joined: Sure, I will write to [email protected] now.< that has no closing > yet, the advisor holds that tail back, waits for the next chunk, and only emits once the placeholder is whole. The hold is capped at the length of the longest possible placeholder (16 characters), so a stray < in ordinary text is released once enough text has followed it, and a final flush on completion emits whatever is still pending. That cap and the flush are read from the code; no test in the module feeds it a stray <.
The two methods that do it (PiiRedactionAdvisor.java):
ChatClientResponse onChunk(ChatClientResponse chunk) {
String text = Texts.text(chunk);
last = chunk;
if (text.isEmpty()) {
return chunk;
}
pending.append(text);
int cut = safeCut();
String emit = session.restore(pending.substring(0, cut));
pending.delete(0, cut);
return Texts.mapResponseText(chunk, t -> emit);
}PiiRedactionAdvisor.java):
private int safeCut() {
int open = pending.lastIndexOf("<");
if (open >= 0 && pending.indexOf(">", open) < 0 && pending.length() - open <= LONGEST_PLACEHOLDER) {
return open;
}
return pending.length();
}
}Going deeper: why this advisor implements the interfaces directly
BaseAdvisor would have been shorter, and wrong. Its after hook sees only the last chunk (the previous section), so it cannot restore text that arrives in pieces. Implementing StreamAdvisor yourself and using map and concatWith on the Flux gives the advisor every chunk, plus a place to emit a final tail chunk when the stream completes. The tail is a fresh response that carries the last chunk’s context but no usage or finish-reason metadata of its own; the model’s own last chunk, which carries them, has already passed through by then.
PiiRedactionAdvisor.java— the whole advisor, includingStreamRestorer.NaiveStreamingPii.java— the deliberately wrong per-chunk version used for the comparison.
Where the redactor sits decides who sees the email
Redaction protects the model, but the model is not the only thing that reads the prompt. Your log does. So does chat memory, which stores the conversation so it can be replayed next turn. Whether they see the raw email or the placeholder is decided by nothing except the order numbers. The test runs the same user message, “My email is [email protected]”, through three arrangements and records what each reader saw. The three arrangements, with the memory advisor fixed at +200 (fromoutput/10-pii-order.txt):
memory advisor is fixed at HIGHEST_PRECEDENCE + 200
user says: My email is [email protected]
A redaction +100, logging +400 (redaction outside both):
log line saw: "My email is <EMAIL_1>"
memory stored: "My email is <EMAIL_1>"
model was sent: "My email is <EMAIL_1>"
B redaction +100, logging +50 (logging outside redaction):
log line saw: "My email is [email protected]"
memory stored: "My email is <EMAIL_1>"
model was sent: "My email is <EMAIL_1>"
C redaction +300 (inside the memory advisor), logging +400:
log line saw: "My email is <EMAIL_1>"
memory stored: "My email is [email protected]"
model was sent: "My email is <EMAIL_1>"The rule. Give the redactor the lowest number of anything that stores, logs or counts text — and lower than Spring AI’s memory advisor. If a new advisor of yours records message text, its number goes above the redactor’s. The constants class in this module is there to make that argument visible in a code review.
Going deeper: the same reasoning for other advisors
The rule generalises. Anything that persists or emits text (a cache, an audit trail, a metrics tag built from the prompt) wants to sit inside the redactor. The exception is an advisor that needs the raw text on purpose, such as one that blocks a request containing a forbidden word before it is rewritten; that one sits outside and must not log. The three arrangements are asserted, not just printed, in
PiiOrderTest.java, so changing a number in Orders.java without thinking about this fails a build.
PiiOrderTest.java— the three arrangements.- Chat memory in Spring AI 2.0 — where the memory advisor’s messages go once stored.
With memory, placeholders have to be numbered per conversation
There is one more way this goes wrong, and it appears only once memory replays earlier turns. The simplest redactor numbers placeholders from one in every request. Turn 1 sends “My email is <EMAIL_1>” and memory stores it. Turn 2 says “Also cc [email protected]”, and the fresh numbering calls Bob<EMAIL_1> too. Now the model is sent a conversation where one label means two different people.
Two turns, first with numbering that restarts on every request, then with numbering kept for the conversation (the default here) (from output/11-pii-multi-turn.txt):
numbering restarts on every request:
model was sent on turn 2: U:My email is <EMAIL_1> | A:You said: My email is <EMAIL_1> | U:Also cc <EMAIL_1>
caller receives: You said: Also cc [email protected]
numbering kept per conversation (the default):
model was sent on turn 2: U:My email is <EMAIL_1> | A:You said: My email is <EMAIL_1> | U:Also cc <EMAIL_2>
caller receives: You said: Also cc [email protected]
what the memory stores (placeholders, never the addresses):
USER My email is <EMAIL_1>
ASSISTANT You said: My email is <EMAIL_1>
USER Also cc <EMAIL_2>
ASSISTANT You said: Also cc <EMAIL_2>Also cc <EMAIL_1> right after My email is <EMAIL_1>: to it, Priya and Bob are one address. The restore on the way out still worked for this turn, which is what makes the bug invisible until a model reasons about who is who. In the second block Bob is <EMAIL_2>. The advisor keeps one Session per conversation ID, taken from the same ChatMemory.CONVERSATION_ID parameter the memory advisor uses (PiiRedactionAdvisor.java). The last lines of the transcript show what memory holds: placeholders, never the addresses.
Two costs of this design. The mapping from placeholder to real value lives in the advisor’s memory, keyed by conversation, so it holds real personal data for as long as the process runs and is lost when the process restarts — after a restart, a placeholder replayed from a persistent chat memory (JDBC or Redis) has nothing to be restored from. The advisor has a forget(conversationId) to drop a conversation’s mapping, but nothing in the module calls it; you would call it when a conversation ends. Both points follow from the code and were not exercised by a test: if you keep chat memory in a database, keep this mapping somewhere equally durable and access-controlled, or accept that restored text is best-effort.
Going deeper: what a placeholder does to the model’s answer
A model given
<EMAIL_1> can still reason about it as an opaque token (“I will send it to that address”), but it cannot do anything that depends on the real value: it cannot check the domain, guess the name from it or write a personalised greeting from it. That is the trade-off the redactor makes, and it is the reason to redact only what the model does not need. I did not measure answer quality with a live model (see the last callout), so this is a design consequence rather than a measured one.
PiiMultiTurnTest.java— both numbering modes side by side.
A token budget that refuses before the model is called
Providers bill by tokens, roughly the number of word-pieces in the text you send and the text you get back. Two things go wrong with cost in practice: one request that is enormous (someone pastes a whole log file), and one user who is not enormous once but never stops. The budget advisor enforces one limit for each. A cap on the estimated size of a single prompt catches the first; a running total per user, built from the token usage the provider reports on each response, catches the second. The important word is before: a refusal costs nothing only if it happens before the model is called. The check runs first, then the call, then the accounting (TokenBudgetAdvisor.java):
public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain) {
String user = userOf(request);
check(user, request);
ChatClientResponse response = chain.nextCall(request);
Usage usage = usageOf(response);
record(user, usage != null ? usage.getTotalTokens() : estimate(request) + estimator.estimate(Texts.text(response)));
return response;
}ifs. The estimate counts every message in the prompt, which at this position in the chain includes whatever the memory advisor added (TokenBudgetAdvisor.java):
private void check(String user, ChatClientRequest request) {
int prompt = estimate(request);
if (prompt > maxPromptTokens) {
throw new TokenBudgetExceededException(
"prompt is about " + prompt + " tokens, the limit per request is " + maxPromptTokens);
}
if (spent(user) >= maxTokensPerUser) {
throw new TokenBudgetExceededException(
"user " + user + " has used " + spent(user) + " of " + maxTokensPerUser + " tokens");
}
}output/12-token-budget.txt):
alice asks short questions; "spent" is the usage the model reported:
call 1: answered, spent=13, model calls=1
call 2: answered, spent=26, model calls=2
call 3: answered, spent=39, model calls=3
call 4: answered, spent=52, model calls=4
call 5: answered, spent=65, model calls=5
call 6: refused (user alice has used 65 of 60 tokens), spent=65, model calls=5
bob pastes a stack trace of 103 estimated tokens:
refused: prompt is about 103 tokens, the limit per request is 40
model calls: 5 (was 5), bob's spent: 0
bob then asks a short question: answered, bob spent=13, alice spent=65output/13-token-budget-stream.txt):
same question, same answer:
call, usage from the response: 9 tokens
stream, usage on the last chunk: 9 tokens
stream, no usage reported (estimated): 9 tokensEstimates are estimates. The third line agrees with the other two only because the stub and the advisor use the same tokenizer (JTokkit’s o200k_base). A real provider counts with its own tokenizer, and a stream that reports no usage may still be billed by the provider. Use the estimate for the per-request cap, where being roughly right is enough, and the provider’s reported usage for anything you would put on an invoice. What the estimate is compared with here is another JTokkit number, not a provider bill.
Going deeper: state, scope and what the budget does not cover
The per-user totals are in a
ConcurrentHashMap of counters inside the advisor, so they are per instance and reset when the process restarts. Two application instances give a user twice the allowance; a real deployment keeps the counters in Redis or a database, and resets them on a schedule. The user comes from a context parameter (budget_user) that the controller sets from the authenticated user — in the demo an X-User header purely so the tests need no login. Do not take it from anything the client can choose. The advisor sits at +250, inside the memory advisor, so the prompt it estimates already includes the replayed history, which is what you want the cap to apply to.
TokenBudgetAdvisor.java— the whole advisor.TokenBudgetTest.java— the tests behind transcripts 12 to 14.- Run LLMs locally with Spring AI and Ollama — a way to try all of this without any token bill.
Outside or inside the tool loop: one line, or one line per round trip
When the model can call tools, one question may cost several model calls: the model asks for a tool, your code runs it and sends the result back, and only then does the model answer. Spring AI runs that loop inside the tool-calling advisor at +300. So an advisor numbered below 300 is outside the loop and is entered once per question, and one numbered above 300 is inside it and is entered once per model call. The test asks a question that needs one tool call, so the model is called twice, and puts the logger on either side. The same logger at +250 and at +400 (fromoutput/15-tool-loop-order.txt):
logging advisor at +250 (outside the tool loop):
[LoggingAdvisor] request messages=1 roles=U
[LoggingAdvisor] response chars=20 took 5 ms
logging advisor at +400 (inside the tool loop):
[LoggingAdvisor] request messages=1 roles=U
[LoggingAdvisor] response chars=0 took 5 ms
[LoggingAdvisor] request messages=3 roles=UAT
[LoggingAdvisor] response chars=20 took 5 msUAT: the user’s question, the assistant’s tool call, and the tool’s result. Neither is right or wrong; it is a choice you make with a number. A logger for “what did the user ask and what did we answer” goes outside the loop. A logger or a counter for “what did we send the provider, and what did each call cost” goes inside it.
That matters most for the budget. A budget advisor outside the loop is entered once and sees the answer that comes out; a budget inside it is entered on every round. The test gives the scripted model a usage of 100+10 tokens for the round that asks for the tool and 130+20 for the round that answers.
The same question, the budget advisor on each side of the loop (from output/21-tool-loop-budget.txt):
the model reports 100+10 tokens for round 1 (asks for the tool) and 130+20 for round 2 (answers)
so the provider would bill 260 tokens for this one question
budget advisor at +250 (outside the tool loop): recorded 260
budget advisor at +400 (inside the tool loop): recorded 260Going deeper: why the numbers in this module are what they are
Orders.java places the budget at +250 and the logger at +400. The budget is outside the loop because its per-user total needs to be recorded once per question from the accumulated usage. The logger is inside because the point of a debugging log is to show each thing sent to the provider. If you want the cap checked on every round as well, put a second, check-only budget advisor at a number above 300. I did not build that second advisor, so treat it as a suggestion the ordering rules support rather than something the tests ran.
ToolLoopOrderTest.javaandToolLoopBudgetTest.java— the two tests above.- Tool calling in Spring AI 2.0 — the loop itself.
When an advisor says no: what the caller sees
An advisor refuses by throwing. That is a plainRuntimeException on a call and an error signal on a stream, and the difference between them is where people lose an afternoon. The test throws the same IllegalStateException("refused") five ways and records at which point the caller finds out.
Where each refusal surfaced (from output/16-error-propagation.txt):
plain advisor, call failed while running it with IllegalStateException: refused
plain advisor, stream, throws eagerly failed while running it with IllegalStateException: refused
plain advisor, stream, error inside Flux.defer failed while running it with IllegalStateException: refused
BaseAdvisor.before, call failed while running it with IllegalStateException: refused
BaseAdvisor.before, stream failed while running it with IllegalStateException: Stream processing failed (cause: IllegalStateException: refused)
model calls: 0.stream().content()) did not throw, for any of the three variants, so a try/catch around the line that starts the stream catches nothing; the error arrives when something subscribes. Whether the advisor threw eagerly or returned an error signal made no visible difference. The last row is the one to remember: an exception thrown from a BaseAdvisor‘s before on a stream reaches the subscriber wrapped in an IllegalStateException with the message “Stream processing failed”, and the original is only the cause. Code that catches your own exception type on a stream will not see it. The budget advisor implements StreamAdvisor directly and raises the error inside the Flux, so a subscriber receives TokenBudgetExceededException itself.
What the subscriber of a refused stream receives (from output/14-token-budget-stream-refusal.txt):
subscriber got: TokenBudgetExceededException
message: prompt is about 13 tokens, the limit per request is 5
model calls: 0Expect an ERROR in the log. When the test above runs, Spring AI’sOver HTTP the call path needs nothing more than a status on the exception, which Spring MVC turns into the response (MessageAggregatorlogsERROR ... Aggregation Errorwith the exception’s stack trace on the console. That is the refusal being reported by the framework, not a failure in your advisor and not something the tests assert on, so it has no transcript; the module’sREADMEnotes it. If your alerting counts ERROR lines, a refused stream will count.
TokenBudgetExceededException.java):
@ResponseStatus(HttpStatus.TOO_MANY_REQUESTS)
public class TokenBudgetExceededException extends RuntimeException {output/19-web-429.txt):
limits for this run: 60 tokens per request, 30 per user
dana request 1 -> HTTP 200, model calls so far: 1
dana request 2 -> HTTP 200, model calls so far: 2
dana request 3 -> HTTP 429, model calls so far: 2
dana request 4 -> HTTP 429, model calls so far: 2
erin pastes a long stack trace -> HTTP 429, model calls: 2 (was 2)Going deeper: making the refusal readable
A bare 429 tells a client to back off but not why. In a real service, add an
@RestControllerAdvice that turns TokenBudgetExceededException into a JSON body with a stable error code, and decide whether the per-request refusal (this prompt is too large; retrying the same text will never work) should be a 4xx different from the per-user refusal (come back later). Neither is in the module: it stops at a status code. The whole application is wired in AdvisorConfig.java, and the error test is ErrorPropagationTest.java.
AdvisorWebTest.java— the HTTP test.- Spring MVC: exception handling.
Testing an advisor without a model, a network or Spring
Everything above was proven without an API key, and the cheapest version of that needs noChatClient at all. An advisor is an ordinary object with one method; the chain it calls is an interface with three. Give it a stub chain that records what reached it and answers with a fixed response, and you can call the advisor directly.
The stub stands in for “everything after me”. It records the request it received, and builds a response from it (UnitTestingWithoutModelTest.java):
private static final class StubChain implements CallAdvisorChain {
final AtomicReference<ChatClientRequest> received = new AtomicReference<>();
@Override
public ChatClientResponse nextCall(ChatClientRequest request) {
received.set(request);
String last = request.prompt().getUserMessage().getText();
return ChatClientResponse.builder()
.chatResponse(ChatResponse.builder()
.generations(List.of(new Generation(new AssistantMessage("Noted: " + last))))
.build())
.build();
}
@Override
public List<CallAdvisor> getCallAdvisors() {
return new ArrayList<>();
}
@Override
public CallAdvisorChain copy(CallAdvisor after) {
return this;
}
}t.line calls write the transcript that follows; the two assertions are the point (UnitTestingWithoutModelTest.java):
PiiRedactionAdvisor advisor = new PiiRedactionAdvisor(Orders.PII_REDACTION, true, false);
StubChain chain = new StubChain();
ChatClientRequest request = ChatClientRequest.builder().prompt(new Prompt("Mail [email protected] please")).build();
ChatClientResponse response = advisor.adviseCall(request, chain);
String forwarded = chain.received.get().prompt().getUserMessage().getText();
String answer = Texts.text(response);
t.line("what the caller sent: %s", request.prompt().getUserMessage().getText());
t.line("what reached the rest: %s", forwarded);
t.line("what the caller got back: %s", answer);
t.line("caller's request unchanged: %s", request.prompt().getUserMessage().getText().contains("[email protected]"));
assertThat(forwarded).isEqualTo("Mail <EMAIL_1> please");
assertThat(answer).isEqualTo("Noted: Mail [email protected] please");output/18-unit-test-no-model.txt):
what the caller sent: Mail [email protected] please
what reached the rest: Mail <EMAIL_1> please
what the caller got back: Noted: Mail [email protected] please
caller's request unchanged: trueChatClient and a chain, which is what the probe advisors do: they record when they are entered and left, and a test asserts the sequence. To check what the model received you need a model stand-in that keeps its prompts; the module’s RecordingModel does that and also streams its answer in chunks of a size you choose, so a chunk-boundary bug is one method call to reproduce. And to check the HTTP behaviour you start the application with that stand-in as the only model, which is what the web test above did.
Three habits that paid off. Print what you measure into a file and assert the same thing, so the article and the build cannot disagree. Test each advisor at the two positions you might put it, because the position is half its behaviour. And test the stream path separately from the call path every time: in this module the two paths differed in thread, in error wrapping and in what after receives.
Going deeper: what the recording stub does and does not fake
RecordingModel.java answers “You said: <last message>” by default, can be given any reply function, reports token usage counted with JTokkit, and on a stream puts the finish reason and usage on the last chunk only — the shape the BaseAdvisor transcript relies on. Flags switch the finish reason and the usage off, to reproduce a provider that omits them. It does not fake latency, retries, rate limits or malformed JSON, and a real provider’s usage fields are its own; use the stub for what the application does with a response, not for what a provider will send.
Probe.java— the probe advisor used for the ordering tests.run-all.sh— runs all 21 tests and regeneratesoutput/.
Should you even do this?
Yes for cross-cutting rules; no for a rule that belongs to one call. An advisor earns its place when the same rule must apply to every model call and you do not trust yourself to remember it: log every call, scrub every prompt, count every token. For a rule that applies to one endpoint, an ordinary method call is simpler and easier to read, and Spring AI already shipsSimpleLoggerAdvisorandSafeGuardAdvisor, so check whether they are enough before writing a third logger. Latency and token counts you intend to graph and alert on belong in metrics and traces, not in log lines; see Micrometer and OpenTelemetry on Spring Boot 4. On the two advisors that look like security or cost controls: a regular-expression redactor is a floor that catches the obvious, not a compliance control, and an in-memory budget protects one instance for one process lifetime. Both are worth having and neither is a reason to stop thinking about where the data goes or who pays for it. Finally, every advisor adds a second place for a bug to hide, and the order numbers are shared global state between advisors written by different people. Keep the list short, keep the numbers in one file, and have a test that prints the chain.
What I did not verify. No live model was called, so nothing here says how a real model reacts to placeholders or how a provider’s billing differs from a JTokkit estimate. Streaming was tested with a scripted chunked model, never over a network stream to a browser, and I did not test a refused stream over HTTP. The budget’s check-then-record is not atomic (two simultaneous requests from one user can both pass the check); that follows from the code and I did not run a concurrent test. The placeholder mapping’s durability and the effect of a restart follow from the code and were not run. Only English-style emails, Luhn-valid card numbers and two phone formats were tested; names, addresses and national ID numbers are explicitly not handled. Nothing was run on a multi-instance deployment, with a distributed budget store, or with a provider other than a stub. Order numbers were read from Spring AI 2.0.1 only.
Further reading
- The code for this article: the
advisorsmodule of asmhatre/spring-ai, with all 21 transcripts underoutput/and its README. - Chat memory in Spring AI 2.0: JDBC, Redis and windowed conversations — the memory advisor at +200 that the redactor has to sit outside.
- Spring AI 2.0 ChatClient on Spring Boot 4.1 and tool calling.
- Spring AI 1.x to 2.0 migration guide on this site.
- Micrometer and OpenTelemetry on Spring Boot 4 for metrics and traces.
- Spring AI reference: Advisors API and Upgrade notes.
- Project Reactor reference, for the
Fluxthe stream advisors return. - OWASP Top 10 for LLM Applications.
- JTokkit, the tokenizer behind the estimates.
No Comments yet!