Add langchain4j-interop: Spring AI 2.0 and LangChain4j 1.22 side by side
Same order assistant in both frameworks against a local fake OpenAI server: starter bean collision, wire differences, tool failure and loop bounds, observability meters, MCP-first tools, and a compile check of the 2025 article's snippets and Maven coordinates. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01G8ikz8xdWuTP5yun8DZ1hk
This commit is contained in:
Claude
@@ -23,5 +23,6 @@ Runnable companion code for the Spring AI articles on [ankurm.com](https://ankur
|
|||||||
| [`providers/`](providers) | One ticket-summarising app on the real `OpenAiChatModel`, `AnthropicChatModel` and `GoogleGenAiChatModel` against a local server in all three wire formats: where each puts the system prompt and what it sends by default, portable versus provider-specific options (and the per-call `ChatOptions` that crashes two providers and silently resets the third), prompt caching, a cost table from the vendors' price sheets, and failover with the retry multiplication measured. No vendor API called; token counts and cache hits are simulated from documented rules; latency not measured. Spring Boot 4.1.1, Spring AI 2.0.1, Java 25. | [Anthropic Claude vs OpenAI vs Gemini in Spring AI 2.0](https://ankurm.com/spring-ai-claude-vs-openai-vs-gemini-switch-providers-compare-cost/) |
|
| [`providers/`](providers) | One ticket-summarising app on the real `OpenAiChatModel`, `AnthropicChatModel` and `GoogleGenAiChatModel` against a local server in all three wire formats: where each puts the system prompt and what it sends by default, portable versus provider-specific options (and the per-call `ChatOptions` that crashes two providers and silently resets the third), prompt caching, a cost table from the vendors' price sheets, and failover with the retry multiplication measured. No vendor API called; token counts and cache hits are simulated from documented rules; latency not measured. Spring Boot 4.1.1, Spring AI 2.0.1, Java 25. | [Anthropic Claude vs OpenAI vs Gemini in Spring AI 2.0](https://ankurm.com/spring-ai-claude-vs-openai-vs-gemini-switch-providers-compare-cost/) |
|
||||||
| [`llm-gateway/`](llm-gateway) | A gateway service on Spring AI 2.0: model-hint routing, failover under the tool-calling advisor (a provider failing mid tool loop does not re-run the tool), one circuit breaker per provider, dollar caps that reserve before the call, a token-per-hour limiter, and a tenant-keyed semantic cache. Real OpenAI and Anthropic models against a local fake; no vendor API called. Spring Boot 4.1.1, Spring AI 2.0.1, Java 25. | [The LLM Gateway Pattern for Java Microservices](https://ankurm.com/llm-gateway-pattern-java-microservices/) |
|
| [`llm-gateway/`](llm-gateway) | A gateway service on Spring AI 2.0: model-hint routing, failover under the tool-calling advisor (a provider failing mid tool loop does not re-run the tool), one circuit breaker per provider, dollar caps that reserve before the call, a token-per-hour limiter, and a tenant-keyed semantic cache. Real OpenAI and Anthropic models against a local fake; no vendor API called. Spring Boot 4.1.1, Spring AI 2.0.1, Java 25. | [The LLM Gateway Pattern for Java Microservices](https://ankurm.com/llm-gateway-pattern-java-microservices/) |
|
||||||
| [`semantic-cache/`](semantic-cache) | A `CallAdvisor` that serves a stored answer when a new question embeds close to an old one, over a Redis Stack vector index and a real in-process embedding model (all-MiniLM-L6-v2). A labelled set of 30 intents x 4 phrasings plus 20 off-topic questions swept across thresholds (hit rate against wrong answers), 600 requests replayed with an illustrative cost model, and the traps reproduced: Spring AI's score is `(1 + cosine) / 2`, reset-password and reset-2FA are close enough to answer for each other, a tenant filter, expiry, and an index built for another embedding model. The chat model is a script that counts calls; one test uses a real local model. Spring Boot 4.1.1, Spring AI 2.0.1, Java 25. | [Semantic Caching for LLM Calls in Spring Boot with Redis Vector Search](https://ankurm.com/semantic-caching-for-llm-calls-in-spring-boot-with-redis-vector-search/) |
|
| [`semantic-cache/`](semantic-cache) | A `CallAdvisor` that serves a stored answer when a new question embeds close to an old one, over a Redis Stack vector index and a real in-process embedding model (all-MiniLM-L6-v2). A labelled set of 30 intents x 4 phrasings plus 20 off-topic questions swept across thresholds (hit rate against wrong answers), 600 requests replayed with an illustrative cost model, and the traps reproduced: Spring AI's score is `(1 + cosine) / 2`, reset-password and reset-2FA are close enough to answer for each other, a tenant filter, expiry, and an index built for another embedding model. The chat model is a script that counts calls; one test uses a real local model. Spring Boot 4.1.1, Spring AI 2.0.1, Java 25. | [Semantic Caching for LLM Calls in Spring Boot with Redis Vector Search](https://ankurm.com/semantic-caching-for-llm-calls-in-spring-boot-with-redis-vector-search/) |
|
||||||
|
| [`langchain4j-interop/`](langchain4j-interop) | The same order assistant in Spring AI 2.0.1 and LangChain4j 1.22.0 in one Spring Boot 4.1.1 app, against a local fake OpenAI server: the bean collision when both OpenAI starters are present, the two requests compared on the wire, a throwing tool and a looping model (what each framework bounds), the meters each records, one MCP tool server used from both, and which lines of the 2025 version of the article still compile. Java 25. | [Spring AI vs LangChain4j in 2026: Is the Pairing Still Worth It?](https://ankurm.com/spring-ai-langchain4j-production-ai-microservices-java/) |
|
||||||
|
|
||||||
Upgrading from Spring AI 1.x: [migration guide](https://ankurm.com/spring-ai-1-to-2-migration-guide/).
|
Upgrading from Spring AI 1.x: [migration guide](https://ankurm.com/spring-ai-1-to-2-migration-guide/).
|
||||||
|
|||||||
@@ -0,0 +1,2 @@
|
|||||||
|
target/
|
||||||
|
*.log
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# langchain4j-interop
|
||||||
|
|
||||||
|
Companion code for [Spring AI vs LangChain4j in 2026: Is the Pairing Still Worth It?](https://ankurm.com/spring-ai-langchain4j-production-ai-microservices-java/), part of the [Spring AI series](../README.md) on ankurm.com.
|
||||||
|
|
||||||
|
One order-support assistant built twice, once with Spring AI 2.0.1 and once with LangChain4j 1.22.0, in one Spring Boot 4.1.1 project, against a local fake OpenAI server (no vendor API is called). The tests answer five questions and `scripts/compile-old-snippets.sh` answers a sixth. Every figure in the article is quoted from a file in [`output/`](output).
|
||||||
|
|
||||||
|
| File | Question |
|
||||||
|
|---|---|
|
||||||
|
| `01-both-starters.txt` | What happens when both OpenAI starters are on the classpath? |
|
||||||
|
| `02-same-assistant.txt` | Same assistant, same question: what differs on the wire? |
|
||||||
|
| `03-tool-throws.txt` | A tool that throws, and a model that will not stop calling it: what bounds the loop? |
|
||||||
|
| `04-observability.txt` | Which meters does each framework record? |
|
||||||
|
| `05-mcp-first.txt` | One MCP tool server consumed from both frameworks |
|
||||||
|
| `06-old-code-vs-current.txt` | Which lines from the original (2025) article still compile, and do its Maven coordinates still exist? |
|
||||||
|
|
||||||
|
## Versions
|
||||||
|
|
||||||
|
Spring Boot 4.1.1, Spring AI 2.0.1, LangChain4j 1.22.0 (`langchain4j-open-ai-spring-boot4-starter` and `langchain4j-spring-boot4-starter` are 1.22.0-beta32, beta only), Java 25.
|
||||||
|
|
||||||
|
## Run
|
||||||
|
|
||||||
|
```bash
|
||||||
|
scripts/run-all.sh # tests -> output/01..05, then the old-snippet check -> output/06
|
||||||
|
```
|
||||||
|
|
||||||
|
No services are needed. Script 06 looks up Maven Central (it retries on rate limiting); `src/broken/` holds the old snippets, one per file, which are compiled outside the main build on purpose.
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
# Both starters in one Spring Boot 4.1.1 application
|
||||||
|
|
||||||
|
Spring AI: spring-ai-starter-model-openai 2.0.1. LangChain4j: langchain4j-open-ai-spring-boot4-starter 1.22.0-beta32.
|
||||||
|
|
||||||
|
1. Both starters, nothing else:
|
||||||
|
FAILED: BeanDefinitionOverrideException: bean 'openAiChatModel' is defined twice, by org.springframework.ai.model.openai.autoconfigure.OpenAiChatAutoConfiguration and by dev.langchain4j.openai.spring.OpenAiAutoConfiguration
|
||||||
|
|
||||||
|
2. spring.autoconfigure.exclude=dev.langchain4j.openai.spring.OpenAiAutoConfiguration:
|
||||||
|
context started
|
||||||
|
bean chatClientBuilder -> org.springframework.ai.chat.client.ChatClient$Builder
|
||||||
|
bean openAiChatModel -> org.springframework.ai.openai.OpenAiChatModel
|
||||||
|
beans of type dev.langchain4j.model.chat.ChatModel: 0
|
||||||
|
beans of type org.springframework.ai.chat.model.ChatModel: 1
|
||||||
|
|
||||||
|
3. spring.main.allow-bean-definition-overriding=true:
|
||||||
|
context started
|
||||||
|
bean openAiChatModel -> org.springframework.ai.openai.OpenAiChatModel
|
||||||
|
bean chatClientBuilder -> org.springframework.ai.chat.client.ChatClient$Builder
|
||||||
|
beans of type dev.langchain4j.model.chat.ChatModel: 0
|
||||||
|
beans of type org.springframework.ai.chat.model.ChatModel: 1
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
# The same order assistant in Spring AI and in LangChain4j, one question: "Where is ORD-1001?"
|
||||||
|
|
||||||
|
Spring AI : ChatClient.builder(model).defaultSystem(..).defaultTools(new SpringAiTools(orders))
|
||||||
|
LangChain4j: AiServices.builder(Support.class).chatModel(model).tools(new Lc4jTools(orders))
|
||||||
|
|
||||||
|
Spring AI answer : Done: "Order ORD-1001: Status=SHIPPED, ETA=2026-10-14"
|
||||||
|
LangChain4j answer : Done: Order ORD-1001: Status=SHIPPED, ETA=2026-10-14
|
||||||
|
HTTP requests sent : Spring AI 2, LangChain4j 2. Tool body executed: Spring AI 1, LangChain4j 1.
|
||||||
|
|
||||||
|
What differs on the wire (computed from the four bodies below):
|
||||||
|
top-level fields : Spring AI [messages, model, tools], LangChain4j [messages, model, stream, tools]
|
||||||
|
tool schema : Spring AI has $schema=true additionalProperties=true; LangChain4j has $schema=false additionalProperties=false
|
||||||
|
tool result message: Spring AI sent "\"Order ORD-1001: Status=SHIPPED, ETA=2026-10-14\"", LangChain4j sent "Order ORD-1001: Status=SHIPPED, ETA=2026-10-14"
|
||||||
|
|
||||||
|
--- Spring AI, request 1
|
||||||
|
fields: [messages, model, tools]
|
||||||
|
system: "You are a customer support agent. Use the tools for order questions. Be concise."
|
||||||
|
user: "Where is ORD-1001?"
|
||||||
|
tool getOrderStatus - Get current order status by order ID params {"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"],"additionalProperties":false}
|
||||||
|
|
||||||
|
--- Spring AI, request 2
|
||||||
|
fields: [messages, model, tools]
|
||||||
|
system: "You are a customer support agent. Use the tools for order questions. Be concise."
|
||||||
|
user: "Where is ORD-1001?"
|
||||||
|
assistant: tool_call getOrderStatus {"orderId":"ORD-1001"}
|
||||||
|
tool: ""Order ORD-1001: Status=SHIPPED, ETA=2026-10-14""
|
||||||
|
tool getOrderStatus - Get current order status by order ID params {"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"],"additionalProperties":false}
|
||||||
|
|
||||||
|
--- LangChain4j, request 1
|
||||||
|
fields: [messages, model, stream, tools]
|
||||||
|
system: "You are a customer support agent. Use the tools for order questions. Be concise."
|
||||||
|
user: "Where is ORD-1001?"
|
||||||
|
tool getOrderStatus - Get current order status by order ID params {"type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"]}
|
||||||
|
|
||||||
|
--- LangChain4j, request 2
|
||||||
|
fields: [messages, model, stream, tools]
|
||||||
|
system: "You are a customer support agent. Use the tools for order questions. Be concise."
|
||||||
|
user: "Where is ORD-1001?"
|
||||||
|
assistant: tool_call getOrderStatus {"orderId":"ORD-1001"}
|
||||||
|
tool: "Order ORD-1001: Status=SHIPPED, ETA=2026-10-14"
|
||||||
|
tool getOrderStatus - Get current order status by order ID params {"type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"]}
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
# A tool that throws (ORD-500 raises IllegalStateException: order database unavailable)
|
||||||
|
|
||||||
|
Case 1: a well-behaved model. It asks for the tool once, then answers from whatever the tool message says.
|
||||||
|
Spring AI : 2 requests, tool body ran 1 time(s). Tool message the model received: "order database unavailable"
|
||||||
|
answer: Done: order database unavailable
|
||||||
|
LangChain4j : 2 requests, tool body ran 1 time(s). Tool message the model received: "order database unavailable"
|
||||||
|
answer: Done: order database unavailable
|
||||||
|
|
||||||
|
Case 2: an insistent model. It asks for the tool again after every result. The server stops it with an HTTP 500 after 30 requests.
|
||||||
|
Spring AI : 31 requests, tool body ran 30 time(s)
|
||||||
|
exception InternalServerException: 500: cutoff reached
|
||||||
|
LangChain4j : 31 requests, tool body ran 30 time(s)
|
||||||
|
exception InternalServerException (root: HttpException): {"error":{"message":"cutoff reached"}}
|
||||||
|
|
||||||
|
Case 3: the same insistent model, each framework asked to stop after 5 tool rounds.
|
||||||
|
Spring AI (ToolCallingAdvisor + our MaxToolRounds(5) checker): 6 requests, tool body ran 5 time(s)
|
||||||
|
answer:
|
||||||
|
LangChain4j (maxSequentialToolsInvocations(5)) : 6 requests, tool body ran 5 time(s)
|
||||||
|
exception RuntimeException: Something is wrong, exceeded 5 tool calling round trips (maxToolCallingRoundTrips)
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
# What each framework records for one question that uses one tool (2 model requests)
|
||||||
|
|
||||||
|
Spring AI 2.0.1: model and client given an ObservationRegistry plus DefaultMeterObservationHandler and ChatModelMeterObservationHandler (the two Spring Boot registers for you).
|
||||||
|
gen_ai.client.operation [TIMER] error=none, gen_ai.operation.name=chat, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.system=openai
|
||||||
|
gen_ai.client.operation.active [LONG_TASK_TIMER] gen_ai.operation.name=chat, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=none, gen_ai.system=openai
|
||||||
|
gen_ai.client.token.usage [COUNTER] gen_ai.operation.name=chat, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.system=openai, gen_ai.token.type=input
|
||||||
|
gen_ai.client.token.usage [COUNTER] gen_ai.operation.name=chat, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.system=openai, gen_ai.token.type=output
|
||||||
|
gen_ai.client.token.usage [COUNTER] gen_ai.operation.name=chat, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.system=openai, gen_ai.token.type=total
|
||||||
|
okhttp.requests [TIMER] error=none, outcome=SUCCESS
|
||||||
|
okhttp.requests.active [LONG_TASK_TIMER] outcome=UNKNOWN
|
||||||
|
spring.ai.advisor [TIMER] error=none, gen_ai.operation.name=framework, gen_ai.system=spring_ai, spring.ai.kind=advisor
|
||||||
|
spring.ai.advisor.active [LONG_TASK_TIMER] gen_ai.operation.name=framework, gen_ai.system=spring_ai, spring.ai.kind=advisor
|
||||||
|
spring.ai.chat.client [TIMER] error=none, gen_ai.operation.name=framework, gen_ai.system=spring_ai, spring.ai.kind=chat_client
|
||||||
|
spring.ai.chat.client.active [LONG_TASK_TIMER] gen_ai.operation.name=framework, gen_ai.system=spring_ai, spring.ai.kind=chat_client
|
||||||
|
spring.ai.tool [TIMER] error=none, gen_ai.operation.name=execute_tool, gen_ai.system=spring_ai, spring.ai.kind=tool_call
|
||||||
|
spring.ai.tool.active [LONG_TASK_TIMER] gen_ai.operation.name=execute_tool, gen_ai.system=spring_ai, spring.ai.kind=tool_call
|
||||||
|
|
||||||
|
LangChain4j 1.22.0, no listener registered: 0 meters.
|
||||||
|
|
||||||
|
LangChain4j 1.22.0 with TracingChatInterceptor, the listener from the original article (it still compiles and runs):
|
||||||
|
llm.request.latency [TIMER] model=gpt-6.1-sol
|
||||||
|
llm.tokens.input [COUNTER] model=gpt-6.1-sol
|
||||||
|
llm.tokens.output [COUNTER] model=gpt-6.1-sol
|
||||||
|
|
||||||
|
LangChain4j 1.22.0 with the shipped MicrometerMetricsChatModelListener (langchain4j-micrometer-metrics 1.22.0-beta32):
|
||||||
|
gen_ai.client.token.usage [DISTRIBUTION_SUMMARY] gen_ai.operation.name=chat, gen_ai.provider.name=openai, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.token.type=input
|
||||||
|
gen_ai.client.token.usage [DISTRIBUTION_SUMMARY] gen_ai.operation.name=chat, gen_ai.provider.name=openai, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.token.type=output
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
# The order tool served once over MCP (Streamable HTTP), used from Spring AI and from LangChain4j
|
||||||
|
|
||||||
|
MCP server: Spring AI 2.0.1 spring-ai-starter-mcp-server-webmvc, one @McpTool method, on port 33237.
|
||||||
|
Spring AI client : McpClient.sync(HttpClientStreamableHttpTransport) -> SyncMcpToolCallbackProvider -> ChatClient.defaultToolCallbacks
|
||||||
|
LangChain4j client: DefaultMcpClient + StreamableHttpMcpTransport -> McpToolProvider -> AiServices.toolProvider
|
||||||
|
|
||||||
|
Spring AI answer : Done: [{"text":"Order ORD-1002: Status=PROCESSING, ETA=2026-10-18"}] (2 requests to the model)
|
||||||
|
tool offered to the model: tool getOrderStatus - Get current order status by order ID params {"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"]}
|
||||||
|
LangChain4j answer : Done: Order ORD-1002: Status=PROCESSING, ETA=2026-10-18 (2 requests to the model)
|
||||||
|
tool offered to the model: tool getOrderStatus - Get current order status by order ID params {"type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"]}
|
||||||
|
|
||||||
|
Times the one tool method on the server ran: 1 for Spring AI, then 1 more for LangChain4j.
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
# The original article's code, compiled against Spring AI 2.0.1 and LangChain4j 1.22.0 (one file per API line)
|
||||||
|
|
||||||
|
OldCallAroundAdvisor: DOES NOT COMPILE
|
||||||
|
OldCallAroundAdvisor.java:1: error: cannot find symbol
|
||||||
|
symbol: class AdvisedRequest
|
||||||
|
location: package org.springframework.ai.chat.client.advisor.api
|
||||||
|
OldCallAroundAdvisor.java:2: error: cannot find symbol
|
||||||
|
symbol: class AdvisedResponse
|
||||||
|
location: package org.springframework.ai.chat.client.advisor.api
|
||||||
|
OldChatService: compiles
|
||||||
|
OldDocumentGetContent: DOES NOT COMPILE
|
||||||
|
OldDocumentGetContent.java:6: error: cannot find symbol
|
||||||
|
symbol: method getContent()
|
||||||
|
location: variable d of type Document
|
||||||
|
OldLangChain4jBuilder: DOES NOT COMPILE
|
||||||
|
OldLangChain4jBuilder.java:9: error: cannot find symbol
|
||||||
|
symbol: method chatLanguageModel(OpenAiChatModel)
|
||||||
|
location: class AiServices<Agent>
|
||||||
|
OldMessageGetContent: DOES NOT COMPILE
|
||||||
|
OldMessageGetContent.java:6: error: cannot find symbol
|
||||||
|
symbol: method getContent()
|
||||||
|
location: variable m of type Message
|
||||||
|
OldMeterRegistryCustomizer: DOES NOT COMPILE
|
||||||
|
OldMeterRegistryCustomizer.java:1: error: package org.springframework.boot.actuate.autoconfigure.metrics does not exist
|
||||||
|
OldMeterRegistryCustomizer.java:5: error: cannot find symbol
|
||||||
|
symbol: class MeterRegistryCustomizer
|
||||||
|
location: class OldMeterRegistryCustomizer
|
||||||
|
OldPdfConfig: compiles
|
||||||
|
OldSearchRequest: DOES NOT COMPILE
|
||||||
|
OldSearchRequest.java:6: error: cannot find symbol
|
||||||
|
symbol: method query(String)
|
||||||
|
location: class SearchRequest
|
||||||
|
OldTokenTextSplitter: DOES NOT COMPILE
|
||||||
|
OldTokenTextSplitter.java:5: error: no suitable constructor found for TokenTextSplitter(int,int,int,int,boolean)
|
||||||
|
|
||||||
|
# The original article's Maven coordinates, looked up on Maven Central
|
||||||
|
|
||||||
|
org/springframework/ai/spring-ai-openai-spring-boot-starter: latest release none, latest of any kind 1.0.0-M6
|
||||||
|
org/springframework/ai/spring-ai-pgvector-store-spring-boot-starter: latest release none, latest of any kind 1.0.0-M6
|
||||||
|
org/springframework/ai/spring-ai-starter-model-openai: latest release 2.0.1, latest of any kind 2.1.0-M1
|
||||||
|
org/springframework/ai/spring-ai-starter-vector-store-pgvector: latest release 2.0.1, latest of any kind 2.1.0-M1
|
||||||
|
dev/langchain4j/langchain4j-spring-boot-starter: latest release 0.36.2, latest of any kind 1.22.0-beta32
|
||||||
|
dev/langchain4j/langchain4j-open-ai: latest release 1.22.0, latest of any kind 1.22.0
|
||||||
|
dev/langchain4j/langchain4j-spring-boot4-starter: latest release none, latest of any kind 1.22.0-beta32
|
||||||
|
|
||||||
|
# Class names the original article used, searched in the jars this build resolves (mvn dependency:build-classpath)
|
||||||
|
|
||||||
|
TokenBudgetAdvisor: not found in any jar
|
||||||
|
RequestResponseAdvisor: not found in any jar
|
||||||
|
CallAroundAdvisor: not found in any jar
|
||||||
|
CallAdvisor: found
|
||||||
|
ToolCallingAdvisor: found
|
||||||
|
AdvisedRequest: not found in any jar
|
||||||
|
ChatClientRequest: found
|
||||||
|
EmbeddingClient: not found in any jar
|
||||||
|
EmbeddingModel: found
|
||||||
@@ -0,0 +1,124 @@
|
|||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||||
|
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||||
|
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||||
|
<modelVersion>4.0.0</modelVersion>
|
||||||
|
|
||||||
|
<parent>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-parent</artifactId>
|
||||||
|
<version>4.1.1</version>
|
||||||
|
<relativePath/>
|
||||||
|
</parent>
|
||||||
|
|
||||||
|
<groupId>com.ankurm</groupId>
|
||||||
|
<artifactId>langchain4j-interop</artifactId>
|
||||||
|
<version>1.0.0</version>
|
||||||
|
<name>langchain4j-interop</name>
|
||||||
|
<description>Spring AI 2.0 and LangChain4j in one Spring Boot 4 application: the same order-assistant in both, what each sends, how each handles a failing tool, observability, and tools served once over MCP.</description>
|
||||||
|
|
||||||
|
<properties>
|
||||||
|
<java.version>25</java.version>
|
||||||
|
<spring-ai.version>2.0.1</spring-ai.version>
|
||||||
|
<langchain4j.version>1.22.0</langchain4j.version>
|
||||||
|
<langchain4j.beta.version>1.22.0-beta32</langchain4j.beta.version>
|
||||||
|
</properties>
|
||||||
|
|
||||||
|
<dependencyManagement>
|
||||||
|
<dependencies>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.ai</groupId>
|
||||||
|
<artifactId>spring-ai-bom</artifactId>
|
||||||
|
<version>${spring-ai.version}</version>
|
||||||
|
<type>pom</type>
|
||||||
|
<scope>import</scope>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>dev.langchain4j</groupId>
|
||||||
|
<artifactId>langchain4j-bom</artifactId>
|
||||||
|
<version>${langchain4j.version}</version>
|
||||||
|
<type>pom</type>
|
||||||
|
<scope>import</scope>
|
||||||
|
</dependency>
|
||||||
|
</dependencies>
|
||||||
|
</dependencyManagement>
|
||||||
|
|
||||||
|
<dependencies>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-webmvc</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-actuator</artifactId>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
|
<!-- Spring AI: the OpenAI model and its auto-configuration -->
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.ai</groupId>
|
||||||
|
<artifactId>spring-ai-starter-model-openai</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.ai</groupId>
|
||||||
|
<artifactId>spring-ai-starter-mcp-client</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.ai</groupId>
|
||||||
|
<artifactId>spring-ai-starter-mcp-server-webmvc</artifactId>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
|
<!-- LangChain4j: the Spring Boot 4 starter family (there is a separate -spring-boot4- artifact set) -->
|
||||||
|
<dependency>
|
||||||
|
<groupId>dev.langchain4j</groupId>
|
||||||
|
<artifactId>langchain4j-open-ai-spring-boot4-starter</artifactId>
|
||||||
|
<version>${langchain4j.beta.version}</version>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>dev.langchain4j</groupId>
|
||||||
|
<artifactId>langchain4j-spring-boot4-starter</artifactId>
|
||||||
|
<version>${langchain4j.beta.version}</version>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>dev.langchain4j</groupId>
|
||||||
|
<artifactId>langchain4j-http-client-jdk</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>dev.langchain4j</groupId>
|
||||||
|
<artifactId>langchain4j-mcp</artifactId>
|
||||||
|
<version>${langchain4j.beta.version}</version>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>dev.langchain4j</groupId>
|
||||||
|
<artifactId>langchain4j-micrometer-metrics</artifactId>
|
||||||
|
<version>${langchain4j.beta.version}</version>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.ai</groupId>
|
||||||
|
<artifactId>spring-ai-vector-store</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<!-- Only so scripts/compile-old-snippets.sh has the PDF reader on its classpath. -->
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.ai</groupId>
|
||||||
|
<artifactId>spring-ai-pdf-document-reader</artifactId>
|
||||||
|
<scope>test</scope>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-test</artifactId>
|
||||||
|
<scope>test</scope>
|
||||||
|
</dependency>
|
||||||
|
</dependencies>
|
||||||
|
|
||||||
|
<build>
|
||||||
|
<plugins>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-surefire-plugin</artifactId>
|
||||||
|
<configuration>
|
||||||
|
<argLine>-Duser.timezone=UTC -Dstdout.encoding=UTF-8 -Dfile.encoding=UTF-8</argLine>
|
||||||
|
</configuration>
|
||||||
|
</plugin>
|
||||||
|
</plugins>
|
||||||
|
</build>
|
||||||
|
</project>
|
||||||
+61
@@ -0,0 +1,61 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Compiles each API line of the original (2025-era) article, one file each, against today's Spring AI 2.0.1 and
|
||||||
|
# LangChain4j 1.22.0 jars, and records what the compiler says. Also looks up the original Maven coordinates.
|
||||||
|
set -uo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
OUT=output/06-old-code-vs-current.txt
|
||||||
|
mvn -q -B dependency:build-classpath -Dmdep.outputFile=target/cp.txt -Dmdep.includeScope=test >/dev/null 2>&1
|
||||||
|
CP="$(cat target/cp.txt)"
|
||||||
|
mkdir -p target/old
|
||||||
|
{
|
||||||
|
echo "# The original article's code, compiled against Spring AI 2.0.1 and LangChain4j 1.22.0 (one file per API line)"
|
||||||
|
echo
|
||||||
|
for f in src/broken/Old*.java; do
|
||||||
|
n=$(basename "$f" .java)
|
||||||
|
rm -rf target/old/$n; mkdir -p target/old/$n
|
||||||
|
if err=$(javac -proc:none --release 25 -cp "$CP" -d target/old/$n "$f" 2>&1 | grep -v '^Picked up' | grep -v '^Note:'); then :; fi
|
||||||
|
if [ -z "$err" ]; then
|
||||||
|
echo "$n: compiles"
|
||||||
|
else
|
||||||
|
echo "$n: DOES NOT COMPILE"
|
||||||
|
echo "$err" | grep -E "error:|symbol:|location:" | head -6 | sed 's#^src/broken/##; s/^/ /'
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo
|
||||||
|
echo "# The original article's Maven coordinates, looked up on Maven Central"
|
||||||
|
echo
|
||||||
|
meta() { # groupPath artifact
|
||||||
|
for try in 1 2 3 4 5 6; do
|
||||||
|
code=$(curl -s -o target/meta.xml -w '%{http_code}' "https://repo1.maven.org/maven2/$1/$2/maven-metadata.xml" || echo 000)
|
||||||
|
case "$code" in 200|404) break;; esac
|
||||||
|
sleep $((try * 8))
|
||||||
|
done
|
||||||
|
if [ "$code" = "404" ]; then echo "$1/$2: no such artifact"; return; fi
|
||||||
|
if [ "$code" != "200" ]; then echo "$1/$2: lookup failed (HTTP $code)"; return; fi
|
||||||
|
all=$(tr '\n' ' ' < target/meta.xml | grep -o '<version>[^<]*</version>' | sed 's/<[^>]*>//g')
|
||||||
|
ga=$(echo "$all" | grep -v -E 'M[0-9]|RC|beta|alpha|SNAPSHOT' | tail -1)
|
||||||
|
last=$(echo "$all" | tail -1)
|
||||||
|
echo "$1/$2: latest release ${ga:-none}, latest of any kind $last"
|
||||||
|
}
|
||||||
|
meta org/springframework/ai spring-ai-openai-spring-boot-starter
|
||||||
|
meta org/springframework/ai spring-ai-pgvector-store-spring-boot-starter
|
||||||
|
meta org/springframework/ai spring-ai-starter-model-openai
|
||||||
|
meta org/springframework/ai spring-ai-starter-vector-store-pgvector
|
||||||
|
meta dev/langchain4j langchain4j-spring-boot-starter
|
||||||
|
meta dev/langchain4j langchain4j-open-ai
|
||||||
|
meta dev/langchain4j langchain4j-spring-boot4-starter
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "# Class names the original article used, searched in the jars this build resolves (mvn dependency:build-classpath)"
|
||||||
|
echo
|
||||||
|
mvn -q -B dependency:build-classpath -Dmdep.outputFile=target/cp.txt >/dev/null 2>&1
|
||||||
|
for name in TokenBudgetAdvisor RequestResponseAdvisor CallAroundAdvisor CallAdvisor ToolCallingAdvisor AdvisedRequest ChatClientRequest EmbeddingClient EmbeddingModel; do
|
||||||
|
hits=0
|
||||||
|
for j in $(tr ':' ' ' < target/cp.txt); do
|
||||||
|
listing=$(unzip -l "$j" 2>/dev/null || true)
|
||||||
|
if grep -q "/$name\.class" <<< "$listing"; then hits=$((hits+1)); fi
|
||||||
|
done
|
||||||
|
if [ "$hits" -gt 0 ]; then echo "$name: found"; else echo "$name: not found in any jar"; fi
|
||||||
|
done
|
||||||
|
} > "$OUT"
|
||||||
|
cat "$OUT"
|
||||||
Executable
+8
@@ -0,0 +1,8 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Regenerates every file under output/ (01-05 from the tests, 06 from the old-snippet script).
|
||||||
|
# No services needed: the model is a local fake OpenAI server inside the tests.
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
mvn -q -B test 2>&1 | grep -E "Tests run:|BUILD|FAIL|ERROR" || true
|
||||||
|
bash scripts/compile-old-snippets.sh
|
||||||
|
ls output
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
import org.springframework.ai.chat.client.advisor.api.AdvisedRequest;
|
||||||
|
import org.springframework.ai.chat.client.advisor.api.AdvisedResponse;
|
||||||
|
import org.springframework.ai.chat.client.advisor.api.CallAroundAdvisor;
|
||||||
|
import org.springframework.ai.chat.client.advisor.api.CallAroundAdvisorChain;
|
||||||
|
|
||||||
|
// From the original TokenBudgetAdvisor.
|
||||||
|
class OldCallAroundAdvisor implements CallAroundAdvisor {
|
||||||
|
@Override
|
||||||
|
public AdvisedResponse aroundCall(AdvisedRequest request, CallAroundAdvisorChain chain) {
|
||||||
|
return chain.nextAroundCall(request);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public String getName() { return "x"; }
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public int getOrder() { return 0; }
|
||||||
|
}
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
import org.springframework.ai.chat.client.ChatClient;
|
||||||
|
import reactor.core.publisher.Flux;
|
||||||
|
|
||||||
|
// The original ChatService, unchanged: this one is expected to still compile.
|
||||||
|
class OldChatService {
|
||||||
|
private final ChatClient chatClient;
|
||||||
|
|
||||||
|
OldChatService(ChatClient.Builder builder) {
|
||||||
|
this.chatClient = builder
|
||||||
|
.defaultSystem("You are a senior Java architect. Be precise and concise.")
|
||||||
|
.build();
|
||||||
|
}
|
||||||
|
|
||||||
|
public String ask(String userMessage) {
|
||||||
|
return chatClient.prompt().user(userMessage).call().content();
|
||||||
|
}
|
||||||
|
|
||||||
|
public Flux<String> stream(String userMessage) {
|
||||||
|
return chatClient.prompt().user(userMessage).stream().content();
|
||||||
|
}
|
||||||
|
|
||||||
|
public ArchitectureReview reviewCode(String codeSnippet) {
|
||||||
|
return chatClient.prompt()
|
||||||
|
.user(u -> u.text("Review this Java code for production readiness: {code}").param("code", codeSnippet))
|
||||||
|
.call()
|
||||||
|
.entity(ArchitectureReview.class);
|
||||||
|
}
|
||||||
|
|
||||||
|
public record ArchitectureReview(String verdict, List<String> issues, List<String> suggestions, int score) { }
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
import org.springframework.ai.document.Document;
|
||||||
|
|
||||||
|
// From the original RAGService: Document::getContent
|
||||||
|
class OldDocumentGetContent {
|
||||||
|
String text(Document d) {
|
||||||
|
return d.getContent();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
import dev.langchain4j.model.openai.OpenAiChatModel;
|
||||||
|
import dev.langchain4j.service.AiServices;
|
||||||
|
|
||||||
|
// From the original AgentConfig.
|
||||||
|
class OldLangChain4jBuilder {
|
||||||
|
interface Agent { String chat(String m); }
|
||||||
|
|
||||||
|
Agent agent(OpenAiChatModel model) {
|
||||||
|
return AiServices.builder(Agent.class).chatLanguageModel(model).build();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
import org.springframework.ai.chat.messages.Message;
|
||||||
|
|
||||||
|
// From the original TokenBudgetAdvisor.estimateTokens: m.getContent()
|
||||||
|
class OldMessageGetContent {
|
||||||
|
int length(Message m) {
|
||||||
|
return m.getContent().length();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
import org.springframework.boot.actuate.autoconfigure.metrics.MeterRegistryCustomizer;
|
||||||
|
|
||||||
|
// From the original ObservabilityConfig.
|
||||||
|
class OldMeterRegistryCustomizer {
|
||||||
|
MeterRegistryCustomizer<?> unused;
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
import org.springframework.ai.reader.pdf.config.PdfDocumentReaderConfig;
|
||||||
|
|
||||||
|
// From the original DocumentIngestionService.
|
||||||
|
class OldPdfConfig {
|
||||||
|
Object config() {
|
||||||
|
return PdfDocumentReaderConfig.builder().withPageTopMargin(0).build();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
import org.springframework.ai.vectorstore.SearchRequest;
|
||||||
|
|
||||||
|
// From the original RAGService and SemanticCacheService.
|
||||||
|
class OldSearchRequest {
|
||||||
|
Object build(String question, int topK) {
|
||||||
|
return SearchRequest.query(question).withTopK(topK).withSimilarityThreshold(0.75);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
import org.springframework.ai.transformer.splitter.TokenTextSplitter;
|
||||||
|
|
||||||
|
// From the original DocumentIngestionService: this one is expected to still compile.
|
||||||
|
class OldTokenTextSplitter {
|
||||||
|
TokenTextSplitter splitter = new TokenTextSplitter(512, 50, 5, 10000, true);
|
||||||
|
}
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import dev.langchain4j.agent.tool.P;
|
||||||
|
import dev.langchain4j.agent.tool.Tool;
|
||||||
|
|
||||||
|
/** The same tool as LangChain4j sees it: same body, different annotations. */
|
||||||
|
public class Lc4jTools {
|
||||||
|
|
||||||
|
private final OrderService orders;
|
||||||
|
|
||||||
|
public Lc4jTools(OrderService orders) {
|
||||||
|
this.orders = orders;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Tool("Get current order status by order ID")
|
||||||
|
public String getOrderStatus(@P("The unique identifier of the order") String orderId) {
|
||||||
|
return orders.describe(orderId);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import java.util.concurrent.atomic.AtomicInteger;
|
||||||
|
|
||||||
|
import org.springframework.ai.chat.model.ChatResponse;
|
||||||
|
import org.springframework.ai.model.tool.ToolExecutionEligibilityChecker;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A round limit for Spring AI's tool loop. The advisor has no "max iterations" setting; what it has is this hook,
|
||||||
|
* asked once per model reply whether the tool calls in it should be executed. One instance counts one call,
|
||||||
|
* so build one per request.
|
||||||
|
*/
|
||||||
|
public final class MaxToolRounds implements ToolExecutionEligibilityChecker {
|
||||||
|
|
||||||
|
private final int limit;
|
||||||
|
private final AtomicInteger rounds = new AtomicInteger();
|
||||||
|
|
||||||
|
public MaxToolRounds(int limit) {
|
||||||
|
this.limit = limit;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public Boolean apply(ChatResponse response) {
|
||||||
|
return response != null && response.hasToolCalls() && rounds.incrementAndGet() <= limit;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Optional;
|
||||||
|
import java.util.concurrent.atomic.AtomicInteger;
|
||||||
|
|
||||||
|
/** The service both frameworks' tools call. Plain Java: it knows nothing about either framework. */
|
||||||
|
public class OrderService {
|
||||||
|
|
||||||
|
public record Order(String id, String status, String eta) { }
|
||||||
|
|
||||||
|
private final Map<String, Order> orders = Map.of(
|
||||||
|
"ORD-1001", new Order("ORD-1001", "SHIPPED", "2026-10-14"),
|
||||||
|
"ORD-1002", new Order("ORD-1002", "PROCESSING", "2026-10-18"));
|
||||||
|
|
||||||
|
public final AtomicInteger calls = new AtomicInteger();
|
||||||
|
|
||||||
|
public Optional<Order> find(String id) {
|
||||||
|
calls.incrementAndGet();
|
||||||
|
if (id.equals("ORD-500")) {
|
||||||
|
throw new IllegalStateException("order database unavailable");
|
||||||
|
}
|
||||||
|
return Optional.ofNullable(orders.get(id));
|
||||||
|
}
|
||||||
|
|
||||||
|
public String describe(String id) {
|
||||||
|
return find(id).map(o -> "Order %s: Status=%s, ETA=%s".formatted(o.id(), o.status(), o.eta()))
|
||||||
|
.orElse("Order not found: " + id);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import org.springframework.ai.tool.annotation.Tool;
|
||||||
|
import org.springframework.ai.tool.annotation.ToolParam;
|
||||||
|
|
||||||
|
/** The order tool as Spring AI sees it. */
|
||||||
|
public class SpringAiTools {
|
||||||
|
|
||||||
|
private final OrderService orders;
|
||||||
|
|
||||||
|
public SpringAiTools(OrderService orders) {
|
||||||
|
this.orders = orders;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Tool(description = "Get current order status by order ID")
|
||||||
|
public String getOrderStatus(@ToolParam(description = "The unique identifier of the order") String orderId) {
|
||||||
|
return orders.describe(orderId);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import dev.langchain4j.service.SystemMessage;
|
||||||
|
|
||||||
|
/** The LangChain4j side of the assistant: an interface whose implementation LangChain4j generates. */
|
||||||
|
public interface Support {
|
||||||
|
|
||||||
|
@SystemMessage("You are a customer support agent. Use the tools for order questions. Be concise.")
|
||||||
|
String chat(String userMessage);
|
||||||
|
}
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import java.util.concurrent.TimeUnit;
|
||||||
|
|
||||||
|
import dev.langchain4j.model.chat.listener.ChatModelErrorContext;
|
||||||
|
import dev.langchain4j.model.chat.listener.ChatModelListener;
|
||||||
|
import dev.langchain4j.model.chat.listener.ChatModelRequestContext;
|
||||||
|
import dev.langchain4j.model.chat.listener.ChatModelResponseContext;
|
||||||
|
import dev.langchain4j.model.output.TokenUsage;
|
||||||
|
import io.micrometer.core.instrument.MeterRegistry;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The listener from the original version of this article, with Lombok and the Spring annotations removed.
|
||||||
|
* It is the same logic: a timer for latency, counters for input and output tokens, a counter for errors.
|
||||||
|
*/
|
||||||
|
public class TracingChatInterceptor implements ChatModelListener {
|
||||||
|
|
||||||
|
private final MeterRegistry meterRegistry;
|
||||||
|
|
||||||
|
private static final String START_TIME_KEY = "start_time";
|
||||||
|
|
||||||
|
public TracingChatInterceptor(MeterRegistry meterRegistry) {
|
||||||
|
this.meterRegistry = meterRegistry;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void onRequest(ChatModelRequestContext ctx) {
|
||||||
|
ctx.attributes().put(START_TIME_KEY, System.currentTimeMillis());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void onResponse(ChatModelResponseContext ctx) {
|
||||||
|
Object startTimeObj = ctx.attributes().get(START_TIME_KEY);
|
||||||
|
if (!(startTimeObj instanceof Long startTime)) return;
|
||||||
|
|
||||||
|
long latencyMs = System.currentTimeMillis() - startTime;
|
||||||
|
TokenUsage usage = ctx.chatResponse().metadata().tokenUsage();
|
||||||
|
String modelName = ctx.chatRequest().parameters().modelName();
|
||||||
|
|
||||||
|
meterRegistry.timer("llm.request.latency", "model", modelName).record(latencyMs, TimeUnit.MILLISECONDS);
|
||||||
|
if (usage != null) {
|
||||||
|
meterRegistry.counter("llm.tokens.input", "model", modelName).increment(usage.inputTokenCount());
|
||||||
|
meterRegistry.counter("llm.tokens.output", "model", modelName).increment(usage.outputTokenCount());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void onError(ChatModelErrorContext ctx) {
|
||||||
|
meterRegistry.counter("llm.request.errors", "model", ctx.chatRequest().parameters().modelName()).increment();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import org.springframework.ai.chat.client.ChatClient;
|
||||||
|
import org.springframework.ai.openai.OpenAiChatModel;
|
||||||
|
import org.springframework.ai.openai.OpenAiChatOptions;
|
||||||
|
|
||||||
|
import dev.langchain4j.service.AiServices;
|
||||||
|
|
||||||
|
/** The two frameworks pointed at the same server, each wired by hand so nothing is auto-configured. */
|
||||||
|
public final class Wiring {
|
||||||
|
|
||||||
|
public static final String SYSTEM = "You are a customer support agent. Use the tools for order questions. Be concise.";
|
||||||
|
|
||||||
|
private Wiring() { }
|
||||||
|
|
||||||
|
public static OpenAiChatModel springModel(String baseUrl) {
|
||||||
|
return OpenAiChatModel.builder().options(OpenAiChatOptions.builder()
|
||||||
|
.baseUrl(baseUrl + "/v1").apiKey("k").model("gpt-6.1-sol").maxRetries(0).build()).build();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* With both langchain4j-http-client-jdk and the Spring starter's RestClient adapter on the classpath, building a
|
||||||
|
* model by hand without naming an HTTP client throws; the explicit builder below avoids it.
|
||||||
|
*/
|
||||||
|
public static dev.langchain4j.model.openai.OpenAiChatModel lc4jModel(String baseUrl) {
|
||||||
|
return dev.langchain4j.model.openai.OpenAiChatModel.builder()
|
||||||
|
.httpClientBuilder(dev.langchain4j.http.client.jdk.JdkHttpClient.builder())
|
||||||
|
.baseUrl(baseUrl + "/v1").apiKey("k").modelName("gpt-6.1-sol").maxRetries(0).build();
|
||||||
|
}
|
||||||
|
|
||||||
|
public static ChatClient springAssistant(String baseUrl, OrderService orders) {
|
||||||
|
return ChatClient.builder(springModel(baseUrl)).defaultSystem(SYSTEM).defaultTools(new SpringAiTools(orders)).build();
|
||||||
|
}
|
||||||
|
|
||||||
|
public static Support lc4jAssistant(String baseUrl, OrderService orders) {
|
||||||
|
return AiServices.builder(Support.class).chatModel(lc4jModel(baseUrl)).tools(new Lc4jTools(orders)).build();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import static org.assertj.core.api.Assertions.assertThat;
|
||||||
|
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.regex.Matcher;
|
||||||
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import org.springframework.boot.WebApplicationType;
|
||||||
|
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
|
||||||
|
import org.springframework.context.annotation.Configuration;
|
||||||
|
import org.springframework.boot.builder.SpringApplicationBuilder;
|
||||||
|
import org.springframework.context.ConfigurableApplicationContext;
|
||||||
|
|
||||||
|
class BothStartersTest {
|
||||||
|
|
||||||
|
@Configuration
|
||||||
|
@EnableAutoConfiguration
|
||||||
|
static class BothApp { }
|
||||||
|
|
||||||
|
record Outcome(String result, List<String> beans) { }
|
||||||
|
|
||||||
|
static Outcome start(FakeOpenAi fake, String... extra) {
|
||||||
|
List<String> props = new ArrayList<>(List.of(
|
||||||
|
"spring.ai.openai.base-url=" + fake.url(), "spring.ai.openai.api-key=k",
|
||||||
|
"spring.ai.openai.chat.options.model=gpt-6.1-sol",
|
||||||
|
"spring.ai.mcp.client.enabled=false", "spring.ai.mcp.server.enabled=false",
|
||||||
|
"langchain4j.open-ai.chat-model.base-url=" + fake.url() + "/v1",
|
||||||
|
"langchain4j.open-ai.chat-model.api-key=k",
|
||||||
|
"langchain4j.open-ai.chat-model.model-name=gpt-6.1-sol"));
|
||||||
|
props.addAll(List.of(extra));
|
||||||
|
try (ConfigurableApplicationContext ctx = new SpringApplicationBuilder(BothApp.class)
|
||||||
|
.web(WebApplicationType.NONE).properties(props.toArray(String[]::new)).run()) {
|
||||||
|
List<String> beans = new ArrayList<>();
|
||||||
|
for (String n : ctx.getBeanDefinitionNames()) {
|
||||||
|
Class<?> c = null;
|
||||||
|
try {
|
||||||
|
c = ctx.getType(n);
|
||||||
|
} catch (Exception ignored) {
|
||||||
|
// not resolvable without creating it
|
||||||
|
}
|
||||||
|
if (c != null && (c.getName().endsWith("OpenAiChatModel") || c.getName().endsWith("ChatClient$Builder")))
|
||||||
|
beans.add(n + " -> " + c.getName());
|
||||||
|
}
|
||||||
|
beans.add("(count) beans of type dev.langchain4j.model.chat.ChatModel: " + ctx.getBeanNamesForType(dev.langchain4j.model.chat.ChatModel.class).length);
|
||||||
|
beans.add("(count) beans of type org.springframework.ai.chat.model.ChatModel: " + ctx.getBeanNamesForType(org.springframework.ai.chat.model.ChatModel.class).length);
|
||||||
|
return new Outcome("context started", beans);
|
||||||
|
} catch (Exception e) {
|
||||||
|
Throwable root = e;
|
||||||
|
while (root.getCause() != null) root = root.getCause();
|
||||||
|
String msg = root.getMessage();
|
||||||
|
if (root.getClass().getSimpleName().equals("BeanDefinitionOverrideException")) {
|
||||||
|
Matcher m = Pattern.compile("factoryBeanName=([\\w.$]+)").matcher(msg);
|
||||||
|
List<String> owners = new ArrayList<>();
|
||||||
|
while (m.find()) owners.add(m.group(1));
|
||||||
|
msg = "bean 'openAiChatModel' is defined twice, by " + String.join(" and by ", owners);
|
||||||
|
}
|
||||||
|
return new Outcome("FAILED: " + root.getClass().getSimpleName() + ": " + msg, List.of());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void bothStartersInOneApplication() throws Exception {
|
||||||
|
try (FakeOpenAi fake = new FakeOpenAi()) {
|
||||||
|
Outcome both = start(fake);
|
||||||
|
Outcome excluded = start(fake, "spring.autoconfigure.exclude=dev.langchain4j.openai.spring.OpenAiAutoConfiguration");
|
||||||
|
Outcome overriding = start(fake, "spring.main.allow-bean-definition-overriding=true");
|
||||||
|
try (var t = new Transcript("01-both-starters.txt", "Both starters in one Spring Boot 4.1.1 application")) {
|
||||||
|
t.line("Spring AI: spring-ai-starter-model-openai 2.0.1. LangChain4j: langchain4j-open-ai-spring-boot4-starter 1.22.0-beta32.");
|
||||||
|
t.blank();
|
||||||
|
t.line("1. Both starters, nothing else:");
|
||||||
|
t.line(" %s", both.result());
|
||||||
|
t.blank();
|
||||||
|
t.line("2. spring.autoconfigure.exclude=dev.langchain4j.openai.spring.OpenAiAutoConfiguration:");
|
||||||
|
t.line(" %s", excluded.result());
|
||||||
|
excluded.beans().forEach(b -> t.line(" %s", b.startsWith("(count)") ? b.substring(8) : "bean " + b));
|
||||||
|
t.blank();
|
||||||
|
t.line("3. spring.main.allow-bean-definition-overriding=true:");
|
||||||
|
t.line(" %s", overriding.result());
|
||||||
|
overriding.beans().forEach(b -> t.line(" %s", b.startsWith("(count)") ? b.substring(8) : "bean " + b));
|
||||||
|
}
|
||||||
|
assertThat(both.result()).startsWith("FAILED: BeanDefinitionOverrideException");
|
||||||
|
assertThat(excluded.result()).isEqualTo("context started");
|
||||||
|
assertThat(overriding.result()).isEqualTo("context started");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.io.OutputStream;
|
||||||
|
import java.net.InetSocketAddress;
|
||||||
|
import java.nio.charset.StandardCharsets;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.concurrent.CopyOnWriteArrayList;
|
||||||
|
import java.util.regex.Matcher;
|
||||||
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
|
import com.sun.net.httpserver.HttpExchange;
|
||||||
|
import com.sun.net.httpserver.HttpServer;
|
||||||
|
import tools.jackson.databind.JsonNode;
|
||||||
|
import tools.jackson.databind.json.JsonMapper;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A local server speaking the OpenAI chat-completions wire format, so the REAL Spring AI and LangChain4j OpenAI
|
||||||
|
* models talk to it. It is a rule, not a language model: if the last message is a tool result it answers with
|
||||||
|
* that result; if the user mentions an order id and a tool is on offer it asks for the tool; otherwise it
|
||||||
|
* greets. Every request body is recorded.
|
||||||
|
*/
|
||||||
|
public final class FakeOpenAi implements AutoCloseable {
|
||||||
|
|
||||||
|
public enum Mode {
|
||||||
|
/** Ask for the tool once, then answer from the tool result. */
|
||||||
|
NORMAL,
|
||||||
|
/** Ask for the tool again after every tool result, like a model that never gives up. */
|
||||||
|
INSISTENT
|
||||||
|
}
|
||||||
|
|
||||||
|
private static final JsonMapper JSON = JsonMapper.builder().build();
|
||||||
|
private static final Pattern ORDER = Pattern.compile("ORD-\\d+");
|
||||||
|
|
||||||
|
private final HttpServer server;
|
||||||
|
public final List<String> bodies = new CopyOnWriteArrayList<>();
|
||||||
|
public volatile Mode mode = Mode.NORMAL;
|
||||||
|
/** After this many requests the server answers 500, so a loop that never ends still ends in a test. */
|
||||||
|
public volatile int cutoff = Integer.MAX_VALUE;
|
||||||
|
|
||||||
|
public FakeOpenAi() throws IOException {
|
||||||
|
server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||||
|
server.createContext("/v1/chat/completions", this::serve);
|
||||||
|
server.start();
|
||||||
|
}
|
||||||
|
|
||||||
|
public String url() { return "http://127.0.0.1:" + server.getAddress().getPort(); }
|
||||||
|
|
||||||
|
public void reset() {
|
||||||
|
bodies.clear();
|
||||||
|
mode = Mode.NORMAL;
|
||||||
|
cutoff = Integer.MAX_VALUE;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void serve(HttpExchange ex) throws IOException {
|
||||||
|
String body = new String(ex.getRequestBody().readAllBytes(), StandardCharsets.UTF_8);
|
||||||
|
bodies.add(body);
|
||||||
|
String reply;
|
||||||
|
try {
|
||||||
|
reply = bodies.size() > cutoff ? "{\"error\":{\"message\":\"cutoff reached\"}}" : decide(JSON.readTree(body));
|
||||||
|
} catch (Exception e) {
|
||||||
|
reply = "{\"error\":{\"message\":\"fake could not read the request: " + e.getMessage().replace("\"", "'") + "\"}}";
|
||||||
|
}
|
||||||
|
byte[] out = reply.getBytes(StandardCharsets.UTF_8);
|
||||||
|
ex.getResponseHeaders().add("Content-Type", "application/json");
|
||||||
|
ex.sendResponseHeaders(reply.startsWith("{\"error") ? 500 : 200, out.length);
|
||||||
|
try (OutputStream os = ex.getResponseBody()) {
|
||||||
|
os.write(out);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private String decide(JsonNode req) {
|
||||||
|
JsonNode messages = req.get("messages");
|
||||||
|
JsonNode last = messages.get(messages.size() - 1);
|
||||||
|
boolean toolsOffered = req.has("tools") && req.get("tools").size() > 0;
|
||||||
|
String lastRole = last.get("role").asText();
|
||||||
|
int promptChars = req.toString().length();
|
||||||
|
|
||||||
|
if (lastRole.equals("tool")) {
|
||||||
|
if (mode == Mode.INSISTENT && toolsOffered) {
|
||||||
|
return toolCall(lastUserText(messages), promptChars);
|
||||||
|
}
|
||||||
|
return text("Done: " + last.get("content").asText(), promptChars);
|
||||||
|
}
|
||||||
|
String user = lastUserText(messages);
|
||||||
|
Matcher m = ORDER.matcher(user);
|
||||||
|
if (toolsOffered && m.find()) {
|
||||||
|
return toolCall(user, promptChars);
|
||||||
|
}
|
||||||
|
return text("Hello. I can look up orders.", promptChars);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static String lastUserText(JsonNode messages) {
|
||||||
|
for (int i = messages.size() - 1; i >= 0; i--) {
|
||||||
|
JsonNode m = messages.get(i);
|
||||||
|
if (m.get("role").asText().equals("user")) {
|
||||||
|
JsonNode c = m.get("content");
|
||||||
|
return c.isString() ? c.asString() : c.toString();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
|
||||||
|
private static String toolCall(String user, int promptChars) {
|
||||||
|
Matcher m = ORDER.matcher(user);
|
||||||
|
String id = m.find() ? m.group() : "ORD-0";
|
||||||
|
return "{\"id\":\"chatcmpl-t\",\"object\":\"chat.completion\",\"created\":1700000000,\"model\":\"gpt-6.1-sol\","
|
||||||
|
+ "\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":null,\"tool_calls\":[{\"id\":\"call_1\","
|
||||||
|
+ "\"type\":\"function\",\"function\":{\"name\":\"getOrderStatus\",\"arguments\":\"{\\\"orderId\\\":\\\"" + id + "\\\"}\"}}]},"
|
||||||
|
+ "\"finish_reason\":\"tool_calls\"}],\"usage\":{\"prompt_tokens\":" + promptChars / 4 + ",\"completion_tokens\":12,\"total_tokens\":"
|
||||||
|
+ (promptChars / 4 + 12) + "}}";
|
||||||
|
}
|
||||||
|
|
||||||
|
private static String text(String text, int promptChars) {
|
||||||
|
return "{\"id\":\"chatcmpl-x\",\"object\":\"chat.completion\",\"created\":1700000000,\"model\":\"gpt-6.1-sol\","
|
||||||
|
+ "\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":" + JSON.writeValueAsString(text)
|
||||||
|
+ "},\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":" + promptChars / 4 + ",\"completion_tokens\":"
|
||||||
|
+ Math.max(1, text.length() / 4) + ",\"total_tokens\":" + (promptChars / 4 + Math.max(1, text.length() / 4)) + "}}";
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void close() { server.stop(0); }
|
||||||
|
}
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import static org.assertj.core.api.Assertions.assertThat;
|
||||||
|
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import org.springframework.ai.chat.client.ChatClient;
|
||||||
|
import org.springframework.ai.mcp.SyncMcpToolCallbackProvider;
|
||||||
|
import org.springframework.ai.mcp.annotation.McpTool;
|
||||||
|
import org.springframework.ai.mcp.annotation.McpToolParam;
|
||||||
|
import org.springframework.boot.WebApplicationType;
|
||||||
|
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
|
||||||
|
import org.springframework.boot.builder.SpringApplicationBuilder;
|
||||||
|
import org.springframework.context.ConfigurableApplicationContext;
|
||||||
|
import org.springframework.context.annotation.Bean;
|
||||||
|
import org.springframework.context.annotation.Configuration;
|
||||||
|
|
||||||
|
import dev.langchain4j.mcp.McpToolProvider;
|
||||||
|
import dev.langchain4j.mcp.client.DefaultMcpClient;
|
||||||
|
import dev.langchain4j.mcp.client.transport.http.StreamableHttpMcpTransport;
|
||||||
|
import dev.langchain4j.service.AiServices;
|
||||||
|
import io.modelcontextprotocol.client.McpClient;
|
||||||
|
import io.modelcontextprotocol.client.transport.HttpClientStreamableHttpTransport;
|
||||||
|
|
||||||
|
/** The order tool written once, behind an MCP server, and used from both frameworks as a client. */
|
||||||
|
class McpFirstTest {
|
||||||
|
|
||||||
|
/** Nothing here imports either framework's tool annotation: only MCP's. */
|
||||||
|
public static class McpOrderTools {
|
||||||
|
|
||||||
|
private final OrderService orders;
|
||||||
|
|
||||||
|
public McpOrderTools(OrderService orders) {
|
||||||
|
this.orders = orders;
|
||||||
|
}
|
||||||
|
|
||||||
|
@McpTool(name = "getOrderStatus", description = "Get current order status by order ID")
|
||||||
|
public String getOrderStatus(@McpToolParam(description = "The unique identifier of the order", required = true) String orderId) {
|
||||||
|
return orders.describe(orderId);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Configuration
|
||||||
|
@EnableAutoConfiguration
|
||||||
|
static class ServerApp {
|
||||||
|
|
||||||
|
static final OrderService ORDERS = new OrderService();
|
||||||
|
|
||||||
|
@Bean
|
||||||
|
McpOrderTools mcpOrderTools() {
|
||||||
|
return new McpOrderTools(ORDERS);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void theToolBehindMcpServesBothFrameworks() throws Exception {
|
||||||
|
try (FakeOpenAi fake = new FakeOpenAi();
|
||||||
|
ConfigurableApplicationContext server = new SpringApplicationBuilder(ServerApp.class)
|
||||||
|
.web(WebApplicationType.SERVLET)
|
||||||
|
.properties("server.port=0", "spring.ai.openai.api-key=unused",
|
||||||
|
"spring.ai.mcp.client.enabled=false",
|
||||||
|
"spring.autoconfigure.exclude=dev.langchain4j.openai.spring.OpenAiAutoConfiguration",
|
||||||
|
"spring.ai.mcp.server.name=orders", "spring.ai.mcp.server.version=1.0.0",
|
||||||
|
"spring.ai.mcp.server.protocol=streamable")
|
||||||
|
.run()) {
|
||||||
|
int port = server.getEnvironment().getProperty("local.server.port", Integer.class);
|
||||||
|
String base = "http://127.0.0.1:" + port;
|
||||||
|
|
||||||
|
// Spring AI as an MCP client: the tools arrive as ToolCallbacks.
|
||||||
|
var sdkClient = McpClient.sync(HttpClientStreamableHttpTransport.builder(base).endpoint("/mcp").build())
|
||||||
|
.requestTimeout(Duration.ofSeconds(10)).build();
|
||||||
|
sdkClient.initialize();
|
||||||
|
var springProvider = SyncMcpToolCallbackProvider.builder().mcpClients(sdkClient).build();
|
||||||
|
ChatClient spring = ChatClient.builder(Wiring.springModel(fake.url())).defaultSystem(Wiring.SYSTEM)
|
||||||
|
.defaultToolCallbacks(springProvider).build();
|
||||||
|
String springAnswer = spring.prompt().user("Where is ORD-1002?").call().content();
|
||||||
|
int springRequests = fake.bodies.size();
|
||||||
|
String springTool = SameAssistantTest.summarise(fake.bodies.get(0)).stream().filter(l -> l.contains("tool getOrderStatus")).findFirst().orElse("(no tool offered)");
|
||||||
|
int callsAfterSpring = ServerApp.ORDERS.calls.get();
|
||||||
|
fake.reset();
|
||||||
|
|
||||||
|
// LangChain4j as an MCP client.
|
||||||
|
var lcMcp = DefaultMcpClient.builder().key("orders")
|
||||||
|
.transport(StreamableHttpMcpTransport.builder().url(base + "/mcp").build()).build();
|
||||||
|
Support lc = AiServices.builder(Support.class).chatModel(Wiring.lc4jModel(fake.url()))
|
||||||
|
.toolProvider(McpToolProvider.builder().mcpClients(List.of(lcMcp)).build()).build();
|
||||||
|
String lcAnswer = lc.chat("Where is ORD-1002?");
|
||||||
|
int lcRequests = fake.bodies.size();
|
||||||
|
String lcTool = SameAssistantTest.summarise(fake.bodies.get(0)).stream().filter(l -> l.contains("tool getOrderStatus")).findFirst().orElse("(no tool offered)");
|
||||||
|
int callsAfterBoth = ServerApp.ORDERS.calls.get();
|
||||||
|
|
||||||
|
try (var t = new Transcript("05-mcp-first.txt", "The order tool served once over MCP (Streamable HTTP), used from Spring AI and from LangChain4j")) {
|
||||||
|
t.line("MCP server: Spring AI 2.0.1 spring-ai-starter-mcp-server-webmvc, one @McpTool method, on port %d.", port);
|
||||||
|
t.line("Spring AI client : McpClient.sync(HttpClientStreamableHttpTransport) -> SyncMcpToolCallbackProvider -> ChatClient.defaultToolCallbacks");
|
||||||
|
t.line("LangChain4j client: DefaultMcpClient + StreamableHttpMcpTransport -> McpToolProvider -> AiServices.toolProvider");
|
||||||
|
t.blank();
|
||||||
|
t.line("Spring AI answer : %s (%d requests to the model)", springAnswer, springRequests);
|
||||||
|
t.line(" tool offered to the model: %s", springTool);
|
||||||
|
t.line("LangChain4j answer : %s (%d requests to the model)", lcAnswer, lcRequests);
|
||||||
|
t.line(" tool offered to the model: %s", lcTool);
|
||||||
|
t.blank();
|
||||||
|
t.line("Times the one tool method on the server ran: %d for Spring AI, then %d more for LangChain4j.", callsAfterSpring, callsAfterBoth - callsAfterSpring);
|
||||||
|
}
|
||||||
|
sdkClient.close();
|
||||||
|
lcMcp.close();
|
||||||
|
assertThat(springAnswer).contains("PROCESSING").isNotNull();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import static org.assertj.core.api.Assertions.assertThat;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.TreeSet;
|
||||||
|
import java.util.stream.Collectors;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import org.springframework.ai.chat.client.ChatClient;
|
||||||
|
import org.springframework.ai.openai.OpenAiChatModel;
|
||||||
|
import org.springframework.ai.openai.OpenAiChatOptions;
|
||||||
|
|
||||||
|
import dev.langchain4j.micrometer.metrics.listeners.MicrometerMetricsChatModelListener;
|
||||||
|
import dev.langchain4j.service.AiServices;
|
||||||
|
import io.micrometer.core.instrument.Meter;
|
||||||
|
import io.micrometer.core.instrument.MeterRegistry;
|
||||||
|
import io.micrometer.core.instrument.simple.SimpleMeterRegistry;
|
||||||
|
import io.micrometer.observation.ObservationRegistry;
|
||||||
|
import io.micrometer.core.instrument.observation.DefaultMeterObservationHandler;
|
||||||
|
|
||||||
|
class ObservabilityTest {
|
||||||
|
|
||||||
|
static List<String> describe(MeterRegistry r) {
|
||||||
|
return r.getMeters().stream().map(m -> {
|
||||||
|
Meter.Id id = m.getId();
|
||||||
|
String tags = id.getTags().stream().filter(t -> List.of("gen_ai.token.type", "gen_ai.request.model", "gen_ai.response.model",
|
||||||
|
"gen_ai.operation.name", "gen_ai.provider.name", "spring.ai.kind", "model", "gen_ai.system", "tool.name", "error", "outcome")
|
||||||
|
.contains(t.getKey())).map(t -> t.getKey() + "=" + t.getValue()).sorted().collect(Collectors.joining(", "));
|
||||||
|
return id.getName() + " [" + id.getType() + "]" + (tags.isEmpty() ? "" : " " + tags);
|
||||||
|
}).distinct().sorted().toList();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void whatEachFrameworkRecordsWithoutBeingAsked() throws Exception {
|
||||||
|
try (FakeOpenAi fake = new FakeOpenAi()) {
|
||||||
|
// Spring AI: a model and a client that share an ObservationRegistry; a handler turns observations into meters.
|
||||||
|
SimpleMeterRegistry springMeters = new SimpleMeterRegistry();
|
||||||
|
ObservationRegistry observations = ObservationRegistry.create();
|
||||||
|
observations.observationConfig().observationHandler(new DefaultMeterObservationHandler(springMeters));
|
||||||
|
// the handler Spring Boot registers for you; it turns the model's token usage into a meter
|
||||||
|
observations.observationConfig().observationHandler(new org.springframework.ai.chat.observation.ChatModelMeterObservationHandler(springMeters));
|
||||||
|
OrderService sOrders = new OrderService();
|
||||||
|
OpenAiChatModel model = OpenAiChatModel.builder().observationRegistry(observations).options(OpenAiChatOptions.builder()
|
||||||
|
.baseUrl(fake.url() + "/v1").apiKey("k").model("gpt-6.1-sol").maxRetries(0).build()).build();
|
||||||
|
ChatClient spring = ChatClient.builder(model, observations, null, null).defaultSystem(Wiring.SYSTEM)
|
||||||
|
.defaultTools(new SpringAiTools(sOrders)).build();
|
||||||
|
spring.prompt().user("Where is ORD-1001?").call().content();
|
||||||
|
|
||||||
|
// LangChain4j: nothing is recorded unless a listener is registered on the model.
|
||||||
|
SimpleMeterRegistry bare = new SimpleMeterRegistry();
|
||||||
|
AiServices.builder(Support.class).chatModel(Wiring.lc4jModel(fake.url())).tools(new Lc4jTools(new OrderService())).build()
|
||||||
|
.chat("Where is ORD-1001?");
|
||||||
|
List<String> lcNone = describe(bare);
|
||||||
|
|
||||||
|
SimpleMeterRegistry lcOwn = new SimpleMeterRegistry(), lcShipped = new SimpleMeterRegistry();
|
||||||
|
var withOwn = dev.langchain4j.model.openai.OpenAiChatModel.builder()
|
||||||
|
.httpClientBuilder(dev.langchain4j.http.client.jdk.JdkHttpClient.builder())
|
||||||
|
.baseUrl(fake.url() + "/v1").apiKey("k").modelName("gpt-6.1-sol")
|
||||||
|
.listeners(List.of(new TracingChatInterceptor(lcOwn))).build();
|
||||||
|
AiServices.builder(Support.class).chatModel(withOwn).tools(new Lc4jTools(new OrderService())).build().chat("Where is ORD-1001?");
|
||||||
|
var withShipped = dev.langchain4j.model.openai.OpenAiChatModel.builder()
|
||||||
|
.httpClientBuilder(dev.langchain4j.http.client.jdk.JdkHttpClient.builder())
|
||||||
|
.baseUrl(fake.url() + "/v1").apiKey("k").modelName("gpt-6.1-sol")
|
||||||
|
.listeners(List.of(new MicrometerMetricsChatModelListener(lcShipped))).build();
|
||||||
|
AiServices.builder(Support.class).chatModel(withShipped).tools(new Lc4jTools(new OrderService())).build().chat("Where is ORD-1001?");
|
||||||
|
|
||||||
|
try (var t = new Transcript("04-observability.txt", "What each framework records for one question that uses one tool (2 model requests)")) {
|
||||||
|
t.line("Spring AI 2.0.1: model and client given an ObservationRegistry plus DefaultMeterObservationHandler and ChatModelMeterObservationHandler (the two Spring Boot registers for you).");
|
||||||
|
describe(springMeters).forEach(l -> t.line(" %s", l));
|
||||||
|
t.blank();
|
||||||
|
t.line("LangChain4j 1.22.0, no listener registered: %d meters.", lcNone.size());
|
||||||
|
t.blank();
|
||||||
|
t.line("LangChain4j 1.22.0 with TracingChatInterceptor, the listener from the original article (it still compiles and runs):");
|
||||||
|
describe(lcOwn).forEach(l -> t.line(" %s", l));
|
||||||
|
t.blank();
|
||||||
|
t.line("LangChain4j 1.22.0 with the shipped MicrometerMetricsChatModelListener (langchain4j-micrometer-metrics 1.22.0-beta32):");
|
||||||
|
describe(lcShipped).forEach(l -> t.line(" %s", l));
|
||||||
|
}
|
||||||
|
assertThat(springMeters.getMeters()).isNotEmpty();
|
||||||
|
assertThat(lcNone).isEmpty();
|
||||||
|
assertThat(lcOwn.getMeters()).isNotEmpty();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,97 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import static org.assertj.core.api.Assertions.assertThat;
|
||||||
|
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.TreeSet;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import tools.jackson.databind.JsonNode;
|
||||||
|
import tools.jackson.databind.json.JsonMapper;
|
||||||
|
|
||||||
|
class SameAssistantTest {
|
||||||
|
|
||||||
|
static final JsonMapper JSON = JsonMapper.builder().build();
|
||||||
|
|
||||||
|
/** One request body reduced to what a reader compares: top-level fields, messages and tools. */
|
||||||
|
static List<String> summarise(String body) {
|
||||||
|
JsonNode req = JSON.readTree(body);
|
||||||
|
List<String> out = new ArrayList<>();
|
||||||
|
TreeSet<String> keys = new TreeSet<>();
|
||||||
|
req.propertyNames().forEach(keys::add);
|
||||||
|
out.add("fields: " + keys);
|
||||||
|
for (JsonNode m : req.get("messages")) {
|
||||||
|
String role = m.get("role").asText();
|
||||||
|
String detail;
|
||||||
|
if (m.has("tool_calls") && m.get("tool_calls").size() > 0) {
|
||||||
|
JsonNode f = m.get("tool_calls").get(0).get("function");
|
||||||
|
detail = "tool_call " + f.get("name").asText() + " " + f.get("arguments").asText();
|
||||||
|
} else {
|
||||||
|
JsonNode c = m.get("content");
|
||||||
|
detail = c == null || c.isNull() ? "" : c.isString() ? "\"" + c.asString() + "\"" : c.toString();
|
||||||
|
}
|
||||||
|
out.add(" " + role + ": " + detail);
|
||||||
|
}
|
||||||
|
if (req.has("tools")) {
|
||||||
|
for (JsonNode tool : req.get("tools")) {
|
||||||
|
JsonNode f = tool.get("function");
|
||||||
|
out.add(" tool " + f.get("name").asText() + " - " + f.path("description").asText("(no description)") + " params " + f.path("parameters"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
static String names(JsonNode n) {
|
||||||
|
TreeSet<String> k = new TreeSet<>();
|
||||||
|
n.propertyNames().forEach(k::add);
|
||||||
|
return k.toString();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void theSameOrderAssistantInBothFrameworks() throws Exception {
|
||||||
|
try (FakeOpenAi fake = new FakeOpenAi()) {
|
||||||
|
OrderService springOrders = new OrderService(), lcOrders = new OrderService();
|
||||||
|
String springAnswer = Wiring.springAssistant(fake.url(), springOrders).prompt().user("Where is ORD-1001?").call().content();
|
||||||
|
List<String> springBodies = new ArrayList<>(fake.bodies);
|
||||||
|
fake.reset();
|
||||||
|
String lcAnswer = Wiring.lc4jAssistant(fake.url(), lcOrders).chat("Where is ORD-1001?");
|
||||||
|
List<String> lcBodies = new ArrayList<>(fake.bodies);
|
||||||
|
|
||||||
|
try (var t = new Transcript("02-same-assistant.txt", "The same order assistant in Spring AI and in LangChain4j, one question: \"Where is ORD-1001?\"")) {
|
||||||
|
t.line("Spring AI : ChatClient.builder(model).defaultSystem(..).defaultTools(new SpringAiTools(orders))");
|
||||||
|
t.line("LangChain4j: AiServices.builder(Support.class).chatModel(model).tools(new Lc4jTools(orders))");
|
||||||
|
t.blank();
|
||||||
|
t.line("Spring AI answer : %s", springAnswer);
|
||||||
|
t.line("LangChain4j answer : %s", lcAnswer);
|
||||||
|
t.line("HTTP requests sent : Spring AI %d, LangChain4j %d. Tool body executed: Spring AI %d, LangChain4j %d.",
|
||||||
|
springBodies.size(), lcBodies.size(), springOrders.calls.get(), lcOrders.calls.get());
|
||||||
|
t.blank();
|
||||||
|
t.line("What differs on the wire (computed from the four bodies below):");
|
||||||
|
JsonNode s1 = JSON.readTree(springBodies.get(0)), l1 = JSON.readTree(lcBodies.get(0));
|
||||||
|
JsonNode s2 = JSON.readTree(springBodies.get(1)), l2 = JSON.readTree(lcBodies.get(1));
|
||||||
|
t.line(" top-level fields : Spring AI %s, LangChain4j %s", names(s1), names(l1));
|
||||||
|
t.line(" tool schema : Spring AI has $schema=%s additionalProperties=%s; LangChain4j has $schema=%s additionalProperties=%s",
|
||||||
|
s1.at("/tools/0/function/parameters").has("$schema"), s1.at("/tools/0/function/parameters").has("additionalProperties"),
|
||||||
|
l1.at("/tools/0/function/parameters").has("$schema"), l1.at("/tools/0/function/parameters").has("additionalProperties"));
|
||||||
|
t.line(" tool result message: Spring AI sent %s, LangChain4j sent %s", s2.at("/messages/3/content"), l2.at("/messages/3/content"));
|
||||||
|
for (int i = 0; i < springBodies.size(); i++) {
|
||||||
|
t.blank();
|
||||||
|
t.line("--- Spring AI, request %d", i + 1);
|
||||||
|
summarise(springBodies.get(i)).forEach(l -> t.line("%s", l));
|
||||||
|
}
|
||||||
|
for (int i = 0; i < lcBodies.size(); i++) {
|
||||||
|
t.blank();
|
||||||
|
t.line("--- LangChain4j, request %d", i + 1);
|
||||||
|
summarise(lcBodies.get(i)).forEach(l -> t.line("%s", l));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assertThat(springAnswer.replace("\"", "")).isEqualTo(lcAnswer);
|
||||||
|
assertThat(springAnswer).isNotEqualTo(lcAnswer);
|
||||||
|
assertThat(springBodies).hasSize(2);
|
||||||
|
assertThat(lcBodies).hasSize(2);
|
||||||
|
assertThat(springOrders.calls.get()).isEqualTo(1);
|
||||||
|
assertThat(lcOrders.calls.get()).isEqualTo(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import static org.assertj.core.api.Assertions.assertThat;
|
||||||
|
|
||||||
|
import java.util.function.Supplier;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import tools.jackson.databind.JsonNode;
|
||||||
|
import tools.jackson.databind.json.JsonMapper;
|
||||||
|
|
||||||
|
class ToolFailureTest {
|
||||||
|
|
||||||
|
static final JsonMapper JSON = JsonMapper.builder().build();
|
||||||
|
|
||||||
|
record Run(String result, int requests, int toolBodies, String toolMessage) { }
|
||||||
|
|
||||||
|
static Run run(FakeOpenAi fake, OrderService orders, Supplier<String> call) {
|
||||||
|
fake.bodies.clear();
|
||||||
|
orders.calls.set(0);
|
||||||
|
String result;
|
||||||
|
try {
|
||||||
|
result = "answer: " + call.get();
|
||||||
|
} catch (Exception e) {
|
||||||
|
Throwable root = e;
|
||||||
|
while (root.getCause() != null) root = root.getCause();
|
||||||
|
result = "exception " + e.getClass().getSimpleName() + (root != e ? " (root: " + root.getClass().getSimpleName() + ")" : "")
|
||||||
|
+ ": " + String.valueOf(root.getMessage()).lines().findFirst().orElse("");
|
||||||
|
}
|
||||||
|
String toolMsg = "(none sent)";
|
||||||
|
for (String b : fake.bodies) {
|
||||||
|
JsonNode msgs = JSON.readTree(b).get("messages");
|
||||||
|
for (JsonNode m : msgs) if (m.get("role").asText().equals("tool")) toolMsg = m.get("content").toString();
|
||||||
|
}
|
||||||
|
return new Run(result, fake.bodies.size(), orders.calls.get(), toolMsg);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void whatEachFrameworkDoesWhenAToolThrows() throws Exception {
|
||||||
|
try (FakeOpenAi fake = new FakeOpenAi()) {
|
||||||
|
OrderService sOrders = new OrderService(), lOrders = new OrderService();
|
||||||
|
var spring = Wiring.springAssistant(fake.url(), sOrders);
|
||||||
|
var lc = Wiring.lc4jAssistant(fake.url(), lOrders);
|
||||||
|
|
||||||
|
Run s1 = run(fake, sOrders, () -> spring.prompt().user("Where is ORD-500?").call().content());
|
||||||
|
Run l1 = run(fake, lOrders, () -> lc.chat("Where is ORD-500?"));
|
||||||
|
|
||||||
|
// A model that keeps asking for the tool after every result, stopped by the server after 30 requests.
|
||||||
|
fake.mode = FakeOpenAi.Mode.INSISTENT;
|
||||||
|
fake.cutoff = 30;
|
||||||
|
Run s2 = run(fake, sOrders, () -> spring.prompt().user("Where is ORD-500?").call().content());
|
||||||
|
Run l2 = run(fake, lOrders, () -> lc.chat("Where is ORD-500?"));
|
||||||
|
// The same insistent model, now with each framework's own way of capping the rounds at 5.
|
||||||
|
fake.cutoff = 30;
|
||||||
|
var cappedSpring = org.springframework.ai.chat.client.ChatClient.builder(Wiring.springModel(fake.url()))
|
||||||
|
.defaultSystem(Wiring.SYSTEM).defaultTools(new SpringAiTools(sOrders))
|
||||||
|
.defaultAdvisors(org.springframework.ai.chat.client.advisor.ToolCallingAdvisor.builder()
|
||||||
|
.toolExecutionEligibilityChecker(new MaxToolRounds(5)).build()).build();
|
||||||
|
var cappedLc = dev.langchain4j.service.AiServices.builder(Support.class).chatModel(Wiring.lc4jModel(fake.url()))
|
||||||
|
.tools(new Lc4jTools(lOrders)).maxSequentialToolsInvocations(5).build();
|
||||||
|
Run s3 = run(fake, sOrders, () -> String.valueOf(cappedSpring.prompt().user("Where is ORD-500?").call().content()));
|
||||||
|
Run l3 = run(fake, lOrders, () -> cappedLc.chat("Where is ORD-500?"));
|
||||||
|
fake.reset();
|
||||||
|
|
||||||
|
try (var t = new Transcript("03-tool-throws.txt", "A tool that throws (ORD-500 raises IllegalStateException: order database unavailable)")) {
|
||||||
|
t.line("Case 1: a well-behaved model. It asks for the tool once, then answers from whatever the tool message says.");
|
||||||
|
t.line(" Spring AI : %d requests, tool body ran %d time(s). Tool message the model received: %s", s1.requests(), s1.toolBodies(), s1.toolMessage());
|
||||||
|
t.line(" %s", s1.result());
|
||||||
|
t.line(" LangChain4j : %d requests, tool body ran %d time(s). Tool message the model received: %s", l1.requests(), l1.toolBodies(), l1.toolMessage());
|
||||||
|
t.line(" %s", l1.result());
|
||||||
|
t.blank();
|
||||||
|
t.line("Case 2: an insistent model. It asks for the tool again after every result. The server stops it with an HTTP 500 after 30 requests.");
|
||||||
|
t.line(" Spring AI : %d requests, tool body ran %d time(s)", s2.requests(), s2.toolBodies());
|
||||||
|
t.line(" %s", s2.result());
|
||||||
|
t.line(" LangChain4j : %d requests, tool body ran %d time(s)", l2.requests(), l2.toolBodies());
|
||||||
|
t.line(" %s", l2.result());
|
||||||
|
t.blank();
|
||||||
|
t.line("Case 3: the same insistent model, each framework asked to stop after 5 tool rounds.");
|
||||||
|
t.line(" Spring AI (ToolCallingAdvisor + our MaxToolRounds(5) checker): %d requests, tool body ran %d time(s)", s3.requests(), s3.toolBodies());
|
||||||
|
t.line(" %s", s3.result());
|
||||||
|
t.line(" LangChain4j (maxSequentialToolsInvocations(5)) : %d requests, tool body ran %d time(s)", l3.requests(), l3.toolBodies());
|
||||||
|
t.line(" %s", l3.result());
|
||||||
|
}
|
||||||
|
assertThat(s1.requests()).isGreaterThan(0);
|
||||||
|
assertThat(l1.requests()).isGreaterThan(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
package com.ankurm.interop;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.io.PrintWriter;
|
||||||
|
import java.io.StringWriter;
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
|
||||||
|
/** Writes a numbered transcript under {@code output/} and echoes it. Every console block in the article is one of these files. */
|
||||||
|
final class Transcript implements AutoCloseable {
|
||||||
|
|
||||||
|
private final Path path;
|
||||||
|
private final StringWriter buffer = new StringWriter();
|
||||||
|
private final PrintWriter out = new PrintWriter(buffer);
|
||||||
|
|
||||||
|
Transcript(String fileName, String title) {
|
||||||
|
this.path = Path.of("output", fileName);
|
||||||
|
out.println("# " + title);
|
||||||
|
out.println();
|
||||||
|
}
|
||||||
|
|
||||||
|
Transcript line(String format, Object... args) {
|
||||||
|
out.println(args.length == 0 ? format : String.format(format, args));
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
Transcript blank() {
|
||||||
|
out.println();
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void close() {
|
||||||
|
out.flush();
|
||||||
|
try {
|
||||||
|
Files.createDirectories(path.getParent());
|
||||||
|
Files.writeString(path, buffer.toString());
|
||||||
|
} catch (IOException e) {
|
||||||
|
throw new IllegalStateException("could not write " + path, e);
|
||||||
|
}
|
||||||
|
System.out.print(buffer);
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user