Add langchain4j-interop: Spring AI 2.0 and LangChain4j 1.22 side by side
Same order assistant in both frameworks against a local fake OpenAI server: starter bean collision, wire differences, tool failure and loop bounds, observability meters, MCP-first tools, and a compile check of the 2025 article's snippets and Maven coordinates. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01G8ikz8xdWuTP5yun8DZ1hk
This commit is contained in:
Claude
@@ -0,0 +1,2 @@
|
||||
target/
|
||||
*.log
|
||||
@@ -0,0 +1,26 @@
|
||||
# langchain4j-interop
|
||||
|
||||
Companion code for [Spring AI vs LangChain4j in 2026: Is the Pairing Still Worth It?](https://ankurm.com/spring-ai-langchain4j-production-ai-microservices-java/), part of the [Spring AI series](../README.md) on ankurm.com.
|
||||
|
||||
One order-support assistant built twice, once with Spring AI 2.0.1 and once with LangChain4j 1.22.0, in one Spring Boot 4.1.1 project, against a local fake OpenAI server (no vendor API is called). The tests answer five questions and `scripts/compile-old-snippets.sh` answers a sixth. Every figure in the article is quoted from a file in [`output/`](output).
|
||||
|
||||
| File | Question |
|
||||
|---|---|
|
||||
| `01-both-starters.txt` | What happens when both OpenAI starters are on the classpath? |
|
||||
| `02-same-assistant.txt` | Same assistant, same question: what differs on the wire? |
|
||||
| `03-tool-throws.txt` | A tool that throws, and a model that will not stop calling it: what bounds the loop? |
|
||||
| `04-observability.txt` | Which meters does each framework record? |
|
||||
| `05-mcp-first.txt` | One MCP tool server consumed from both frameworks |
|
||||
| `06-old-code-vs-current.txt` | Which lines from the original (2025) article still compile, and do its Maven coordinates still exist? |
|
||||
|
||||
## Versions
|
||||
|
||||
Spring Boot 4.1.1, Spring AI 2.0.1, LangChain4j 1.22.0 (`langchain4j-open-ai-spring-boot4-starter` and `langchain4j-spring-boot4-starter` are 1.22.0-beta32, beta only), Java 25.
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
scripts/run-all.sh # tests -> output/01..05, then the old-snippet check -> output/06
|
||||
```
|
||||
|
||||
No services are needed. Script 06 looks up Maven Central (it retries on rate limiting); `src/broken/` holds the old snippets, one per file, which are compiled outside the main build on purpose.
|
||||
@@ -0,0 +1,20 @@
|
||||
# Both starters in one Spring Boot 4.1.1 application
|
||||
|
||||
Spring AI: spring-ai-starter-model-openai 2.0.1. LangChain4j: langchain4j-open-ai-spring-boot4-starter 1.22.0-beta32.
|
||||
|
||||
1. Both starters, nothing else:
|
||||
FAILED: BeanDefinitionOverrideException: bean 'openAiChatModel' is defined twice, by org.springframework.ai.model.openai.autoconfigure.OpenAiChatAutoConfiguration and by dev.langchain4j.openai.spring.OpenAiAutoConfiguration
|
||||
|
||||
2. spring.autoconfigure.exclude=dev.langchain4j.openai.spring.OpenAiAutoConfiguration:
|
||||
context started
|
||||
bean chatClientBuilder -> org.springframework.ai.chat.client.ChatClient$Builder
|
||||
bean openAiChatModel -> org.springframework.ai.openai.OpenAiChatModel
|
||||
beans of type dev.langchain4j.model.chat.ChatModel: 0
|
||||
beans of type org.springframework.ai.chat.model.ChatModel: 1
|
||||
|
||||
3. spring.main.allow-bean-definition-overriding=true:
|
||||
context started
|
||||
bean openAiChatModel -> org.springframework.ai.openai.OpenAiChatModel
|
||||
bean chatClientBuilder -> org.springframework.ai.chat.client.ChatClient$Builder
|
||||
beans of type dev.langchain4j.model.chat.ChatModel: 0
|
||||
beans of type org.springframework.ai.chat.model.ChatModel: 1
|
||||
@@ -0,0 +1,41 @@
|
||||
# The same order assistant in Spring AI and in LangChain4j, one question: "Where is ORD-1001?"
|
||||
|
||||
Spring AI : ChatClient.builder(model).defaultSystem(..).defaultTools(new SpringAiTools(orders))
|
||||
LangChain4j: AiServices.builder(Support.class).chatModel(model).tools(new Lc4jTools(orders))
|
||||
|
||||
Spring AI answer : Done: "Order ORD-1001: Status=SHIPPED, ETA=2026-10-14"
|
||||
LangChain4j answer : Done: Order ORD-1001: Status=SHIPPED, ETA=2026-10-14
|
||||
HTTP requests sent : Spring AI 2, LangChain4j 2. Tool body executed: Spring AI 1, LangChain4j 1.
|
||||
|
||||
What differs on the wire (computed from the four bodies below):
|
||||
top-level fields : Spring AI [messages, model, tools], LangChain4j [messages, model, stream, tools]
|
||||
tool schema : Spring AI has $schema=true additionalProperties=true; LangChain4j has $schema=false additionalProperties=false
|
||||
tool result message: Spring AI sent "\"Order ORD-1001: Status=SHIPPED, ETA=2026-10-14\"", LangChain4j sent "Order ORD-1001: Status=SHIPPED, ETA=2026-10-14"
|
||||
|
||||
--- Spring AI, request 1
|
||||
fields: [messages, model, tools]
|
||||
system: "You are a customer support agent. Use the tools for order questions. Be concise."
|
||||
user: "Where is ORD-1001?"
|
||||
tool getOrderStatus - Get current order status by order ID params {"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"],"additionalProperties":false}
|
||||
|
||||
--- Spring AI, request 2
|
||||
fields: [messages, model, tools]
|
||||
system: "You are a customer support agent. Use the tools for order questions. Be concise."
|
||||
user: "Where is ORD-1001?"
|
||||
assistant: tool_call getOrderStatus {"orderId":"ORD-1001"}
|
||||
tool: ""Order ORD-1001: Status=SHIPPED, ETA=2026-10-14""
|
||||
tool getOrderStatus - Get current order status by order ID params {"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"],"additionalProperties":false}
|
||||
|
||||
--- LangChain4j, request 1
|
||||
fields: [messages, model, stream, tools]
|
||||
system: "You are a customer support agent. Use the tools for order questions. Be concise."
|
||||
user: "Where is ORD-1001?"
|
||||
tool getOrderStatus - Get current order status by order ID params {"type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"]}
|
||||
|
||||
--- LangChain4j, request 2
|
||||
fields: [messages, model, stream, tools]
|
||||
system: "You are a customer support agent. Use the tools for order questions. Be concise."
|
||||
user: "Where is ORD-1001?"
|
||||
assistant: tool_call getOrderStatus {"orderId":"ORD-1001"}
|
||||
tool: "Order ORD-1001: Status=SHIPPED, ETA=2026-10-14"
|
||||
tool getOrderStatus - Get current order status by order ID params {"type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"]}
|
||||
@@ -0,0 +1,19 @@
|
||||
# A tool that throws (ORD-500 raises IllegalStateException: order database unavailable)
|
||||
|
||||
Case 1: a well-behaved model. It asks for the tool once, then answers from whatever the tool message says.
|
||||
Spring AI : 2 requests, tool body ran 1 time(s). Tool message the model received: "order database unavailable"
|
||||
answer: Done: order database unavailable
|
||||
LangChain4j : 2 requests, tool body ran 1 time(s). Tool message the model received: "order database unavailable"
|
||||
answer: Done: order database unavailable
|
||||
|
||||
Case 2: an insistent model. It asks for the tool again after every result. The server stops it with an HTTP 500 after 30 requests.
|
||||
Spring AI : 31 requests, tool body ran 30 time(s)
|
||||
exception InternalServerException: 500: cutoff reached
|
||||
LangChain4j : 31 requests, tool body ran 30 time(s)
|
||||
exception InternalServerException (root: HttpException): {"error":{"message":"cutoff reached"}}
|
||||
|
||||
Case 3: the same insistent model, each framework asked to stop after 5 tool rounds.
|
||||
Spring AI (ToolCallingAdvisor + our MaxToolRounds(5) checker): 6 requests, tool body ran 5 time(s)
|
||||
answer:
|
||||
LangChain4j (maxSequentialToolsInvocations(5)) : 6 requests, tool body ran 5 time(s)
|
||||
exception RuntimeException: Something is wrong, exceeded 5 tool calling round trips (maxToolCallingRoundTrips)
|
||||
@@ -0,0 +1,27 @@
|
||||
# What each framework records for one question that uses one tool (2 model requests)
|
||||
|
||||
Spring AI 2.0.1: model and client given an ObservationRegistry plus DefaultMeterObservationHandler and ChatModelMeterObservationHandler (the two Spring Boot registers for you).
|
||||
gen_ai.client.operation [TIMER] error=none, gen_ai.operation.name=chat, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.system=openai
|
||||
gen_ai.client.operation.active [LONG_TASK_TIMER] gen_ai.operation.name=chat, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=none, gen_ai.system=openai
|
||||
gen_ai.client.token.usage [COUNTER] gen_ai.operation.name=chat, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.system=openai, gen_ai.token.type=input
|
||||
gen_ai.client.token.usage [COUNTER] gen_ai.operation.name=chat, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.system=openai, gen_ai.token.type=output
|
||||
gen_ai.client.token.usage [COUNTER] gen_ai.operation.name=chat, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.system=openai, gen_ai.token.type=total
|
||||
okhttp.requests [TIMER] error=none, outcome=SUCCESS
|
||||
okhttp.requests.active [LONG_TASK_TIMER] outcome=UNKNOWN
|
||||
spring.ai.advisor [TIMER] error=none, gen_ai.operation.name=framework, gen_ai.system=spring_ai, spring.ai.kind=advisor
|
||||
spring.ai.advisor.active [LONG_TASK_TIMER] gen_ai.operation.name=framework, gen_ai.system=spring_ai, spring.ai.kind=advisor
|
||||
spring.ai.chat.client [TIMER] error=none, gen_ai.operation.name=framework, gen_ai.system=spring_ai, spring.ai.kind=chat_client
|
||||
spring.ai.chat.client.active [LONG_TASK_TIMER] gen_ai.operation.name=framework, gen_ai.system=spring_ai, spring.ai.kind=chat_client
|
||||
spring.ai.tool [TIMER] error=none, gen_ai.operation.name=execute_tool, gen_ai.system=spring_ai, spring.ai.kind=tool_call
|
||||
spring.ai.tool.active [LONG_TASK_TIMER] gen_ai.operation.name=execute_tool, gen_ai.system=spring_ai, spring.ai.kind=tool_call
|
||||
|
||||
LangChain4j 1.22.0, no listener registered: 0 meters.
|
||||
|
||||
LangChain4j 1.22.0 with TracingChatInterceptor, the listener from the original article (it still compiles and runs):
|
||||
llm.request.latency [TIMER] model=gpt-6.1-sol
|
||||
llm.tokens.input [COUNTER] model=gpt-6.1-sol
|
||||
llm.tokens.output [COUNTER] model=gpt-6.1-sol
|
||||
|
||||
LangChain4j 1.22.0 with the shipped MicrometerMetricsChatModelListener (langchain4j-micrometer-metrics 1.22.0-beta32):
|
||||
gen_ai.client.token.usage [DISTRIBUTION_SUMMARY] gen_ai.operation.name=chat, gen_ai.provider.name=openai, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.token.type=input
|
||||
gen_ai.client.token.usage [DISTRIBUTION_SUMMARY] gen_ai.operation.name=chat, gen_ai.provider.name=openai, gen_ai.request.model=gpt-6.1-sol, gen_ai.response.model=gpt-6.1-sol, gen_ai.token.type=output
|
||||
@@ -0,0 +1,12 @@
|
||||
# The order tool served once over MCP (Streamable HTTP), used from Spring AI and from LangChain4j
|
||||
|
||||
MCP server: Spring AI 2.0.1 spring-ai-starter-mcp-server-webmvc, one @McpTool method, on port 33237.
|
||||
Spring AI client : McpClient.sync(HttpClientStreamableHttpTransport) -> SyncMcpToolCallbackProvider -> ChatClient.defaultToolCallbacks
|
||||
LangChain4j client: DefaultMcpClient + StreamableHttpMcpTransport -> McpToolProvider -> AiServices.toolProvider
|
||||
|
||||
Spring AI answer : Done: [{"text":"Order ORD-1002: Status=PROCESSING, ETA=2026-10-18"}] (2 requests to the model)
|
||||
tool offered to the model: tool getOrderStatus - Get current order status by order ID params {"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"]}
|
||||
LangChain4j answer : Done: Order ORD-1002: Status=PROCESSING, ETA=2026-10-18 (2 requests to the model)
|
||||
tool offered to the model: tool getOrderStatus - Get current order status by order ID params {"type":"object","properties":{"orderId":{"type":"string","description":"The unique identifier of the order"}},"required":["orderId"]}
|
||||
|
||||
Times the one tool method on the server ran: 1 for Spring AI, then 1 more for LangChain4j.
|
||||
@@ -0,0 +1,56 @@
|
||||
# The original article's code, compiled against Spring AI 2.0.1 and LangChain4j 1.22.0 (one file per API line)
|
||||
|
||||
OldCallAroundAdvisor: DOES NOT COMPILE
|
||||
OldCallAroundAdvisor.java:1: error: cannot find symbol
|
||||
symbol: class AdvisedRequest
|
||||
location: package org.springframework.ai.chat.client.advisor.api
|
||||
OldCallAroundAdvisor.java:2: error: cannot find symbol
|
||||
symbol: class AdvisedResponse
|
||||
location: package org.springframework.ai.chat.client.advisor.api
|
||||
OldChatService: compiles
|
||||
OldDocumentGetContent: DOES NOT COMPILE
|
||||
OldDocumentGetContent.java:6: error: cannot find symbol
|
||||
symbol: method getContent()
|
||||
location: variable d of type Document
|
||||
OldLangChain4jBuilder: DOES NOT COMPILE
|
||||
OldLangChain4jBuilder.java:9: error: cannot find symbol
|
||||
symbol: method chatLanguageModel(OpenAiChatModel)
|
||||
location: class AiServices<Agent>
|
||||
OldMessageGetContent: DOES NOT COMPILE
|
||||
OldMessageGetContent.java:6: error: cannot find symbol
|
||||
symbol: method getContent()
|
||||
location: variable m of type Message
|
||||
OldMeterRegistryCustomizer: DOES NOT COMPILE
|
||||
OldMeterRegistryCustomizer.java:1: error: package org.springframework.boot.actuate.autoconfigure.metrics does not exist
|
||||
OldMeterRegistryCustomizer.java:5: error: cannot find symbol
|
||||
symbol: class MeterRegistryCustomizer
|
||||
location: class OldMeterRegistryCustomizer
|
||||
OldPdfConfig: compiles
|
||||
OldSearchRequest: DOES NOT COMPILE
|
||||
OldSearchRequest.java:6: error: cannot find symbol
|
||||
symbol: method query(String)
|
||||
location: class SearchRequest
|
||||
OldTokenTextSplitter: DOES NOT COMPILE
|
||||
OldTokenTextSplitter.java:5: error: no suitable constructor found for TokenTextSplitter(int,int,int,int,boolean)
|
||||
|
||||
# The original article's Maven coordinates, looked up on Maven Central
|
||||
|
||||
org/springframework/ai/spring-ai-openai-spring-boot-starter: latest release none, latest of any kind 1.0.0-M6
|
||||
org/springframework/ai/spring-ai-pgvector-store-spring-boot-starter: latest release none, latest of any kind 1.0.0-M6
|
||||
org/springframework/ai/spring-ai-starter-model-openai: latest release 2.0.1, latest of any kind 2.1.0-M1
|
||||
org/springframework/ai/spring-ai-starter-vector-store-pgvector: latest release 2.0.1, latest of any kind 2.1.0-M1
|
||||
dev/langchain4j/langchain4j-spring-boot-starter: latest release 0.36.2, latest of any kind 1.22.0-beta32
|
||||
dev/langchain4j/langchain4j-open-ai: latest release 1.22.0, latest of any kind 1.22.0
|
||||
dev/langchain4j/langchain4j-spring-boot4-starter: latest release none, latest of any kind 1.22.0-beta32
|
||||
|
||||
# Class names the original article used, searched in the jars this build resolves (mvn dependency:build-classpath)
|
||||
|
||||
TokenBudgetAdvisor: not found in any jar
|
||||
RequestResponseAdvisor: not found in any jar
|
||||
CallAroundAdvisor: not found in any jar
|
||||
CallAdvisor: found
|
||||
ToolCallingAdvisor: found
|
||||
AdvisedRequest: not found in any jar
|
||||
ChatClientRequest: found
|
||||
EmbeddingClient: not found in any jar
|
||||
EmbeddingModel: found
|
||||
@@ -0,0 +1,124 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-parent</artifactId>
|
||||
<version>4.1.1</version>
|
||||
<relativePath/>
|
||||
</parent>
|
||||
|
||||
<groupId>com.ankurm</groupId>
|
||||
<artifactId>langchain4j-interop</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<name>langchain4j-interop</name>
|
||||
<description>Spring AI 2.0 and LangChain4j in one Spring Boot 4 application: the same order-assistant in both, what each sends, how each handles a failing tool, observability, and tools served once over MCP.</description>
|
||||
|
||||
<properties>
|
||||
<java.version>25</java.version>
|
||||
<spring-ai.version>2.0.1</spring-ai.version>
|
||||
<langchain4j.version>1.22.0</langchain4j.version>
|
||||
<langchain4j.beta.version>1.22.0-beta32</langchain4j.beta.version>
|
||||
</properties>
|
||||
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-bom</artifactId>
|
||||
<version>${spring-ai.version}</version>
|
||||
<type>pom</type>
|
||||
<scope>import</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-bom</artifactId>
|
||||
<version>${langchain4j.version}</version>
|
||||
<type>pom</type>
|
||||
<scope>import</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</dependencyManagement>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-webmvc</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-actuator</artifactId>
|
||||
</dependency>
|
||||
|
||||
<!-- Spring AI: the OpenAI model and its auto-configuration -->
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-starter-model-openai</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-starter-mcp-client</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-starter-mcp-server-webmvc</artifactId>
|
||||
</dependency>
|
||||
|
||||
<!-- LangChain4j: the Spring Boot 4 starter family (there is a separate -spring-boot4- artifact set) -->
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-open-ai-spring-boot4-starter</artifactId>
|
||||
<version>${langchain4j.beta.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-spring-boot4-starter</artifactId>
|
||||
<version>${langchain4j.beta.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-http-client-jdk</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-mcp</artifactId>
|
||||
<version>${langchain4j.beta.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-micrometer-metrics</artifactId>
|
||||
<version>${langchain4j.beta.version}</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-vector-store</artifactId>
|
||||
</dependency>
|
||||
<!-- Only so scripts/compile-old-snippets.sh has the PDF reader on its classpath. -->
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-pdf-document-reader</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-test</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-surefire-plugin</artifactId>
|
||||
<configuration>
|
||||
<argLine>-Duser.timezone=UTC -Dstdout.encoding=UTF-8 -Dfile.encoding=UTF-8</argLine>
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
#!/usr/bin/env bash
|
||||
# Compiles each API line of the original (2025-era) article, one file each, against today's Spring AI 2.0.1 and
|
||||
# LangChain4j 1.22.0 jars, and records what the compiler says. Also looks up the original Maven coordinates.
|
||||
set -uo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
OUT=output/06-old-code-vs-current.txt
|
||||
mvn -q -B dependency:build-classpath -Dmdep.outputFile=target/cp.txt -Dmdep.includeScope=test >/dev/null 2>&1
|
||||
CP="$(cat target/cp.txt)"
|
||||
mkdir -p target/old
|
||||
{
|
||||
echo "# The original article's code, compiled against Spring AI 2.0.1 and LangChain4j 1.22.0 (one file per API line)"
|
||||
echo
|
||||
for f in src/broken/Old*.java; do
|
||||
n=$(basename "$f" .java)
|
||||
rm -rf target/old/$n; mkdir -p target/old/$n
|
||||
if err=$(javac -proc:none --release 25 -cp "$CP" -d target/old/$n "$f" 2>&1 | grep -v '^Picked up' | grep -v '^Note:'); then :; fi
|
||||
if [ -z "$err" ]; then
|
||||
echo "$n: compiles"
|
||||
else
|
||||
echo "$n: DOES NOT COMPILE"
|
||||
echo "$err" | grep -E "error:|symbol:|location:" | head -6 | sed 's#^src/broken/##; s/^/ /'
|
||||
fi
|
||||
done
|
||||
echo
|
||||
echo "# The original article's Maven coordinates, looked up on Maven Central"
|
||||
echo
|
||||
meta() { # groupPath artifact
|
||||
for try in 1 2 3 4 5 6; do
|
||||
code=$(curl -s -o target/meta.xml -w '%{http_code}' "https://repo1.maven.org/maven2/$1/$2/maven-metadata.xml" || echo 000)
|
||||
case "$code" in 200|404) break;; esac
|
||||
sleep $((try * 8))
|
||||
done
|
||||
if [ "$code" = "404" ]; then echo "$1/$2: no such artifact"; return; fi
|
||||
if [ "$code" != "200" ]; then echo "$1/$2: lookup failed (HTTP $code)"; return; fi
|
||||
all=$(tr '\n' ' ' < target/meta.xml | grep -o '<version>[^<]*</version>' | sed 's/<[^>]*>//g')
|
||||
ga=$(echo "$all" | grep -v -E 'M[0-9]|RC|beta|alpha|SNAPSHOT' | tail -1)
|
||||
last=$(echo "$all" | tail -1)
|
||||
echo "$1/$2: latest release ${ga:-none}, latest of any kind $last"
|
||||
}
|
||||
meta org/springframework/ai spring-ai-openai-spring-boot-starter
|
||||
meta org/springframework/ai spring-ai-pgvector-store-spring-boot-starter
|
||||
meta org/springframework/ai spring-ai-starter-model-openai
|
||||
meta org/springframework/ai spring-ai-starter-vector-store-pgvector
|
||||
meta dev/langchain4j langchain4j-spring-boot-starter
|
||||
meta dev/langchain4j langchain4j-open-ai
|
||||
meta dev/langchain4j langchain4j-spring-boot4-starter
|
||||
|
||||
echo
|
||||
echo "# Class names the original article used, searched in the jars this build resolves (mvn dependency:build-classpath)"
|
||||
echo
|
||||
mvn -q -B dependency:build-classpath -Dmdep.outputFile=target/cp.txt >/dev/null 2>&1
|
||||
for name in TokenBudgetAdvisor RequestResponseAdvisor CallAroundAdvisor CallAdvisor ToolCallingAdvisor AdvisedRequest ChatClientRequest EmbeddingClient EmbeddingModel; do
|
||||
hits=0
|
||||
for j in $(tr ':' ' ' < target/cp.txt); do
|
||||
listing=$(unzip -l "$j" 2>/dev/null || true)
|
||||
if grep -q "/$name\.class" <<< "$listing"; then hits=$((hits+1)); fi
|
||||
done
|
||||
if [ "$hits" -gt 0 ]; then echo "$name: found"; else echo "$name: not found in any jar"; fi
|
||||
done
|
||||
} > "$OUT"
|
||||
cat "$OUT"
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
# Regenerates every file under output/ (01-05 from the tests, 06 from the old-snippet script).
|
||||
# No services needed: the model is a local fake OpenAI server inside the tests.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
mvn -q -B test 2>&1 | grep -E "Tests run:|BUILD|FAIL|ERROR" || true
|
||||
bash scripts/compile-old-snippets.sh
|
||||
ls output
|
||||
@@ -0,0 +1,18 @@
|
||||
import org.springframework.ai.chat.client.advisor.api.AdvisedRequest;
|
||||
import org.springframework.ai.chat.client.advisor.api.AdvisedResponse;
|
||||
import org.springframework.ai.chat.client.advisor.api.CallAroundAdvisor;
|
||||
import org.springframework.ai.chat.client.advisor.api.CallAroundAdvisorChain;
|
||||
|
||||
// From the original TokenBudgetAdvisor.
|
||||
class OldCallAroundAdvisor implements CallAroundAdvisor {
|
||||
@Override
|
||||
public AdvisedResponse aroundCall(AdvisedRequest request, CallAroundAdvisorChain chain) {
|
||||
return chain.nextAroundCall(request);
|
||||
}
|
||||
|
||||
@Override
|
||||
public String getName() { return "x"; }
|
||||
|
||||
@Override
|
||||
public int getOrder() { return 0; }
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
import java.util.List;
|
||||
|
||||
import org.springframework.ai.chat.client.ChatClient;
|
||||
import reactor.core.publisher.Flux;
|
||||
|
||||
// The original ChatService, unchanged: this one is expected to still compile.
|
||||
class OldChatService {
|
||||
private final ChatClient chatClient;
|
||||
|
||||
OldChatService(ChatClient.Builder builder) {
|
||||
this.chatClient = builder
|
||||
.defaultSystem("You are a senior Java architect. Be precise and concise.")
|
||||
.build();
|
||||
}
|
||||
|
||||
public String ask(String userMessage) {
|
||||
return chatClient.prompt().user(userMessage).call().content();
|
||||
}
|
||||
|
||||
public Flux<String> stream(String userMessage) {
|
||||
return chatClient.prompt().user(userMessage).stream().content();
|
||||
}
|
||||
|
||||
public ArchitectureReview reviewCode(String codeSnippet) {
|
||||
return chatClient.prompt()
|
||||
.user(u -> u.text("Review this Java code for production readiness: {code}").param("code", codeSnippet))
|
||||
.call()
|
||||
.entity(ArchitectureReview.class);
|
||||
}
|
||||
|
||||
public record ArchitectureReview(String verdict, List<String> issues, List<String> suggestions, int score) { }
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
import org.springframework.ai.document.Document;
|
||||
|
||||
// From the original RAGService: Document::getContent
|
||||
class OldDocumentGetContent {
|
||||
String text(Document d) {
|
||||
return d.getContent();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
import dev.langchain4j.model.openai.OpenAiChatModel;
|
||||
import dev.langchain4j.service.AiServices;
|
||||
|
||||
// From the original AgentConfig.
|
||||
class OldLangChain4jBuilder {
|
||||
interface Agent { String chat(String m); }
|
||||
|
||||
Agent agent(OpenAiChatModel model) {
|
||||
return AiServices.builder(Agent.class).chatLanguageModel(model).build();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
import org.springframework.ai.chat.messages.Message;
|
||||
|
||||
// From the original TokenBudgetAdvisor.estimateTokens: m.getContent()
|
||||
class OldMessageGetContent {
|
||||
int length(Message m) {
|
||||
return m.getContent().length();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
import org.springframework.boot.actuate.autoconfigure.metrics.MeterRegistryCustomizer;
|
||||
|
||||
// From the original ObservabilityConfig.
|
||||
class OldMeterRegistryCustomizer {
|
||||
MeterRegistryCustomizer<?> unused;
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
import org.springframework.ai.reader.pdf.config.PdfDocumentReaderConfig;
|
||||
|
||||
// From the original DocumentIngestionService.
|
||||
class OldPdfConfig {
|
||||
Object config() {
|
||||
return PdfDocumentReaderConfig.builder().withPageTopMargin(0).build();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
import org.springframework.ai.vectorstore.SearchRequest;
|
||||
|
||||
// From the original RAGService and SemanticCacheService.
|
||||
class OldSearchRequest {
|
||||
Object build(String question, int topK) {
|
||||
return SearchRequest.query(question).withTopK(topK).withSimilarityThreshold(0.75);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
import org.springframework.ai.transformer.splitter.TokenTextSplitter;
|
||||
|
||||
// From the original DocumentIngestionService: this one is expected to still compile.
|
||||
class OldTokenTextSplitter {
|
||||
TokenTextSplitter splitter = new TokenTextSplitter(512, 50, 5, 10000, true);
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import dev.langchain4j.agent.tool.P;
|
||||
import dev.langchain4j.agent.tool.Tool;
|
||||
|
||||
/** The same tool as LangChain4j sees it: same body, different annotations. */
|
||||
public class Lc4jTools {
|
||||
|
||||
private final OrderService orders;
|
||||
|
||||
public Lc4jTools(OrderService orders) {
|
||||
this.orders = orders;
|
||||
}
|
||||
|
||||
@Tool("Get current order status by order ID")
|
||||
public String getOrderStatus(@P("The unique identifier of the order") String orderId) {
|
||||
return orders.describe(orderId);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
|
||||
import org.springframework.ai.chat.model.ChatResponse;
|
||||
import org.springframework.ai.model.tool.ToolExecutionEligibilityChecker;
|
||||
|
||||
/**
|
||||
* A round limit for Spring AI's tool loop. The advisor has no "max iterations" setting; what it has is this hook,
|
||||
* asked once per model reply whether the tool calls in it should be executed. One instance counts one call,
|
||||
* so build one per request.
|
||||
*/
|
||||
public final class MaxToolRounds implements ToolExecutionEligibilityChecker {
|
||||
|
||||
private final int limit;
|
||||
private final AtomicInteger rounds = new AtomicInteger();
|
||||
|
||||
public MaxToolRounds(int limit) {
|
||||
this.limit = limit;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Boolean apply(ChatResponse response) {
|
||||
return response != null && response.hasToolCalls() && rounds.incrementAndGet() <= limit;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
|
||||
/** The service both frameworks' tools call. Plain Java: it knows nothing about either framework. */
|
||||
public class OrderService {
|
||||
|
||||
public record Order(String id, String status, String eta) { }
|
||||
|
||||
private final Map<String, Order> orders = Map.of(
|
||||
"ORD-1001", new Order("ORD-1001", "SHIPPED", "2026-10-14"),
|
||||
"ORD-1002", new Order("ORD-1002", "PROCESSING", "2026-10-18"));
|
||||
|
||||
public final AtomicInteger calls = new AtomicInteger();
|
||||
|
||||
public Optional<Order> find(String id) {
|
||||
calls.incrementAndGet();
|
||||
if (id.equals("ORD-500")) {
|
||||
throw new IllegalStateException("order database unavailable");
|
||||
}
|
||||
return Optional.ofNullable(orders.get(id));
|
||||
}
|
||||
|
||||
public String describe(String id) {
|
||||
return find(id).map(o -> "Order %s: Status=%s, ETA=%s".formatted(o.id(), o.status(), o.eta()))
|
||||
.orElse("Order not found: " + id);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import org.springframework.ai.tool.annotation.Tool;
|
||||
import org.springframework.ai.tool.annotation.ToolParam;
|
||||
|
||||
/** The order tool as Spring AI sees it. */
|
||||
public class SpringAiTools {
|
||||
|
||||
private final OrderService orders;
|
||||
|
||||
public SpringAiTools(OrderService orders) {
|
||||
this.orders = orders;
|
||||
}
|
||||
|
||||
@Tool(description = "Get current order status by order ID")
|
||||
public String getOrderStatus(@ToolParam(description = "The unique identifier of the order") String orderId) {
|
||||
return orders.describe(orderId);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import dev.langchain4j.service.SystemMessage;
|
||||
|
||||
/** The LangChain4j side of the assistant: an interface whose implementation LangChain4j generates. */
|
||||
public interface Support {
|
||||
|
||||
@SystemMessage("You are a customer support agent. Use the tools for order questions. Be concise.")
|
||||
String chat(String userMessage);
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
import dev.langchain4j.model.chat.listener.ChatModelErrorContext;
|
||||
import dev.langchain4j.model.chat.listener.ChatModelListener;
|
||||
import dev.langchain4j.model.chat.listener.ChatModelRequestContext;
|
||||
import dev.langchain4j.model.chat.listener.ChatModelResponseContext;
|
||||
import dev.langchain4j.model.output.TokenUsage;
|
||||
import io.micrometer.core.instrument.MeterRegistry;
|
||||
|
||||
/**
|
||||
* The listener from the original version of this article, with Lombok and the Spring annotations removed.
|
||||
* It is the same logic: a timer for latency, counters for input and output tokens, a counter for errors.
|
||||
*/
|
||||
public class TracingChatInterceptor implements ChatModelListener {
|
||||
|
||||
private final MeterRegistry meterRegistry;
|
||||
|
||||
private static final String START_TIME_KEY = "start_time";
|
||||
|
||||
public TracingChatInterceptor(MeterRegistry meterRegistry) {
|
||||
this.meterRegistry = meterRegistry;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onRequest(ChatModelRequestContext ctx) {
|
||||
ctx.attributes().put(START_TIME_KEY, System.currentTimeMillis());
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onResponse(ChatModelResponseContext ctx) {
|
||||
Object startTimeObj = ctx.attributes().get(START_TIME_KEY);
|
||||
if (!(startTimeObj instanceof Long startTime)) return;
|
||||
|
||||
long latencyMs = System.currentTimeMillis() - startTime;
|
||||
TokenUsage usage = ctx.chatResponse().metadata().tokenUsage();
|
||||
String modelName = ctx.chatRequest().parameters().modelName();
|
||||
|
||||
meterRegistry.timer("llm.request.latency", "model", modelName).record(latencyMs, TimeUnit.MILLISECONDS);
|
||||
if (usage != null) {
|
||||
meterRegistry.counter("llm.tokens.input", "model", modelName).increment(usage.inputTokenCount());
|
||||
meterRegistry.counter("llm.tokens.output", "model", modelName).increment(usage.outputTokenCount());
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onError(ChatModelErrorContext ctx) {
|
||||
meterRegistry.counter("llm.request.errors", "model", ctx.chatRequest().parameters().modelName()).increment();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import org.springframework.ai.chat.client.ChatClient;
|
||||
import org.springframework.ai.openai.OpenAiChatModel;
|
||||
import org.springframework.ai.openai.OpenAiChatOptions;
|
||||
|
||||
import dev.langchain4j.service.AiServices;
|
||||
|
||||
/** The two frameworks pointed at the same server, each wired by hand so nothing is auto-configured. */
|
||||
public final class Wiring {
|
||||
|
||||
public static final String SYSTEM = "You are a customer support agent. Use the tools for order questions. Be concise.";
|
||||
|
||||
private Wiring() { }
|
||||
|
||||
public static OpenAiChatModel springModel(String baseUrl) {
|
||||
return OpenAiChatModel.builder().options(OpenAiChatOptions.builder()
|
||||
.baseUrl(baseUrl + "/v1").apiKey("k").model("gpt-6.1-sol").maxRetries(0).build()).build();
|
||||
}
|
||||
|
||||
/**
|
||||
* With both langchain4j-http-client-jdk and the Spring starter's RestClient adapter on the classpath, building a
|
||||
* model by hand without naming an HTTP client throws; the explicit builder below avoids it.
|
||||
*/
|
||||
public static dev.langchain4j.model.openai.OpenAiChatModel lc4jModel(String baseUrl) {
|
||||
return dev.langchain4j.model.openai.OpenAiChatModel.builder()
|
||||
.httpClientBuilder(dev.langchain4j.http.client.jdk.JdkHttpClient.builder())
|
||||
.baseUrl(baseUrl + "/v1").apiKey("k").modelName("gpt-6.1-sol").maxRetries(0).build();
|
||||
}
|
||||
|
||||
public static ChatClient springAssistant(String baseUrl, OrderService orders) {
|
||||
return ChatClient.builder(springModel(baseUrl)).defaultSystem(SYSTEM).defaultTools(new SpringAiTools(orders)).build();
|
||||
}
|
||||
|
||||
public static Support lc4jAssistant(String baseUrl, OrderService orders) {
|
||||
return AiServices.builder(Support.class).chatModel(lc4jModel(baseUrl)).tools(new Lc4jTools(orders)).build();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.boot.WebApplicationType;
|
||||
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import org.springframework.boot.builder.SpringApplicationBuilder;
|
||||
import org.springframework.context.ConfigurableApplicationContext;
|
||||
|
||||
class BothStartersTest {
|
||||
|
||||
@Configuration
|
||||
@EnableAutoConfiguration
|
||||
static class BothApp { }
|
||||
|
||||
record Outcome(String result, List<String> beans) { }
|
||||
|
||||
static Outcome start(FakeOpenAi fake, String... extra) {
|
||||
List<String> props = new ArrayList<>(List.of(
|
||||
"spring.ai.openai.base-url=" + fake.url(), "spring.ai.openai.api-key=k",
|
||||
"spring.ai.openai.chat.options.model=gpt-6.1-sol",
|
||||
"spring.ai.mcp.client.enabled=false", "spring.ai.mcp.server.enabled=false",
|
||||
"langchain4j.open-ai.chat-model.base-url=" + fake.url() + "/v1",
|
||||
"langchain4j.open-ai.chat-model.api-key=k",
|
||||
"langchain4j.open-ai.chat-model.model-name=gpt-6.1-sol"));
|
||||
props.addAll(List.of(extra));
|
||||
try (ConfigurableApplicationContext ctx = new SpringApplicationBuilder(BothApp.class)
|
||||
.web(WebApplicationType.NONE).properties(props.toArray(String[]::new)).run()) {
|
||||
List<String> beans = new ArrayList<>();
|
||||
for (String n : ctx.getBeanDefinitionNames()) {
|
||||
Class<?> c = null;
|
||||
try {
|
||||
c = ctx.getType(n);
|
||||
} catch (Exception ignored) {
|
||||
// not resolvable without creating it
|
||||
}
|
||||
if (c != null && (c.getName().endsWith("OpenAiChatModel") || c.getName().endsWith("ChatClient$Builder")))
|
||||
beans.add(n + " -> " + c.getName());
|
||||
}
|
||||
beans.add("(count) beans of type dev.langchain4j.model.chat.ChatModel: " + ctx.getBeanNamesForType(dev.langchain4j.model.chat.ChatModel.class).length);
|
||||
beans.add("(count) beans of type org.springframework.ai.chat.model.ChatModel: " + ctx.getBeanNamesForType(org.springframework.ai.chat.model.ChatModel.class).length);
|
||||
return new Outcome("context started", beans);
|
||||
} catch (Exception e) {
|
||||
Throwable root = e;
|
||||
while (root.getCause() != null) root = root.getCause();
|
||||
String msg = root.getMessage();
|
||||
if (root.getClass().getSimpleName().equals("BeanDefinitionOverrideException")) {
|
||||
Matcher m = Pattern.compile("factoryBeanName=([\\w.$]+)").matcher(msg);
|
||||
List<String> owners = new ArrayList<>();
|
||||
while (m.find()) owners.add(m.group(1));
|
||||
msg = "bean 'openAiChatModel' is defined twice, by " + String.join(" and by ", owners);
|
||||
}
|
||||
return new Outcome("FAILED: " + root.getClass().getSimpleName() + ": " + msg, List.of());
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void bothStartersInOneApplication() throws Exception {
|
||||
try (FakeOpenAi fake = new FakeOpenAi()) {
|
||||
Outcome both = start(fake);
|
||||
Outcome excluded = start(fake, "spring.autoconfigure.exclude=dev.langchain4j.openai.spring.OpenAiAutoConfiguration");
|
||||
Outcome overriding = start(fake, "spring.main.allow-bean-definition-overriding=true");
|
||||
try (var t = new Transcript("01-both-starters.txt", "Both starters in one Spring Boot 4.1.1 application")) {
|
||||
t.line("Spring AI: spring-ai-starter-model-openai 2.0.1. LangChain4j: langchain4j-open-ai-spring-boot4-starter 1.22.0-beta32.");
|
||||
t.blank();
|
||||
t.line("1. Both starters, nothing else:");
|
||||
t.line(" %s", both.result());
|
||||
t.blank();
|
||||
t.line("2. spring.autoconfigure.exclude=dev.langchain4j.openai.spring.OpenAiAutoConfiguration:");
|
||||
t.line(" %s", excluded.result());
|
||||
excluded.beans().forEach(b -> t.line(" %s", b.startsWith("(count)") ? b.substring(8) : "bean " + b));
|
||||
t.blank();
|
||||
t.line("3. spring.main.allow-bean-definition-overriding=true:");
|
||||
t.line(" %s", overriding.result());
|
||||
overriding.beans().forEach(b -> t.line(" %s", b.startsWith("(count)") ? b.substring(8) : "bean " + b));
|
||||
}
|
||||
assertThat(both.result()).startsWith("FAILED: BeanDefinitionOverrideException");
|
||||
assertThat(excluded.result()).isEqualTo("context started");
|
||||
assertThat(overriding.result()).isEqualTo("context started");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.io.OutputStream;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.List;
|
||||
import java.util.concurrent.CopyOnWriteArrayList;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
import com.sun.net.httpserver.HttpExchange;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
import tools.jackson.databind.JsonNode;
|
||||
import tools.jackson.databind.json.JsonMapper;
|
||||
|
||||
/**
|
||||
* A local server speaking the OpenAI chat-completions wire format, so the REAL Spring AI and LangChain4j OpenAI
|
||||
* models talk to it. It is a rule, not a language model: if the last message is a tool result it answers with
|
||||
* that result; if the user mentions an order id and a tool is on offer it asks for the tool; otherwise it
|
||||
* greets. Every request body is recorded.
|
||||
*/
|
||||
public final class FakeOpenAi implements AutoCloseable {
|
||||
|
||||
public enum Mode {
|
||||
/** Ask for the tool once, then answer from the tool result. */
|
||||
NORMAL,
|
||||
/** Ask for the tool again after every tool result, like a model that never gives up. */
|
||||
INSISTENT
|
||||
}
|
||||
|
||||
private static final JsonMapper JSON = JsonMapper.builder().build();
|
||||
private static final Pattern ORDER = Pattern.compile("ORD-\\d+");
|
||||
|
||||
private final HttpServer server;
|
||||
public final List<String> bodies = new CopyOnWriteArrayList<>();
|
||||
public volatile Mode mode = Mode.NORMAL;
|
||||
/** After this many requests the server answers 500, so a loop that never ends still ends in a test. */
|
||||
public volatile int cutoff = Integer.MAX_VALUE;
|
||||
|
||||
public FakeOpenAi() throws IOException {
|
||||
server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
server.createContext("/v1/chat/completions", this::serve);
|
||||
server.start();
|
||||
}
|
||||
|
||||
public String url() { return "http://127.0.0.1:" + server.getAddress().getPort(); }
|
||||
|
||||
public void reset() {
|
||||
bodies.clear();
|
||||
mode = Mode.NORMAL;
|
||||
cutoff = Integer.MAX_VALUE;
|
||||
}
|
||||
|
||||
private void serve(HttpExchange ex) throws IOException {
|
||||
String body = new String(ex.getRequestBody().readAllBytes(), StandardCharsets.UTF_8);
|
||||
bodies.add(body);
|
||||
String reply;
|
||||
try {
|
||||
reply = bodies.size() > cutoff ? "{\"error\":{\"message\":\"cutoff reached\"}}" : decide(JSON.readTree(body));
|
||||
} catch (Exception e) {
|
||||
reply = "{\"error\":{\"message\":\"fake could not read the request: " + e.getMessage().replace("\"", "'") + "\"}}";
|
||||
}
|
||||
byte[] out = reply.getBytes(StandardCharsets.UTF_8);
|
||||
ex.getResponseHeaders().add("Content-Type", "application/json");
|
||||
ex.sendResponseHeaders(reply.startsWith("{\"error") ? 500 : 200, out.length);
|
||||
try (OutputStream os = ex.getResponseBody()) {
|
||||
os.write(out);
|
||||
}
|
||||
}
|
||||
|
||||
private String decide(JsonNode req) {
|
||||
JsonNode messages = req.get("messages");
|
||||
JsonNode last = messages.get(messages.size() - 1);
|
||||
boolean toolsOffered = req.has("tools") && req.get("tools").size() > 0;
|
||||
String lastRole = last.get("role").asText();
|
||||
int promptChars = req.toString().length();
|
||||
|
||||
if (lastRole.equals("tool")) {
|
||||
if (mode == Mode.INSISTENT && toolsOffered) {
|
||||
return toolCall(lastUserText(messages), promptChars);
|
||||
}
|
||||
return text("Done: " + last.get("content").asText(), promptChars);
|
||||
}
|
||||
String user = lastUserText(messages);
|
||||
Matcher m = ORDER.matcher(user);
|
||||
if (toolsOffered && m.find()) {
|
||||
return toolCall(user, promptChars);
|
||||
}
|
||||
return text("Hello. I can look up orders.", promptChars);
|
||||
}
|
||||
|
||||
private static String lastUserText(JsonNode messages) {
|
||||
for (int i = messages.size() - 1; i >= 0; i--) {
|
||||
JsonNode m = messages.get(i);
|
||||
if (m.get("role").asText().equals("user")) {
|
||||
JsonNode c = m.get("content");
|
||||
return c.isString() ? c.asString() : c.toString();
|
||||
}
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
private static String toolCall(String user, int promptChars) {
|
||||
Matcher m = ORDER.matcher(user);
|
||||
String id = m.find() ? m.group() : "ORD-0";
|
||||
return "{\"id\":\"chatcmpl-t\",\"object\":\"chat.completion\",\"created\":1700000000,\"model\":\"gpt-6.1-sol\","
|
||||
+ "\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":null,\"tool_calls\":[{\"id\":\"call_1\","
|
||||
+ "\"type\":\"function\",\"function\":{\"name\":\"getOrderStatus\",\"arguments\":\"{\\\"orderId\\\":\\\"" + id + "\\\"}\"}}]},"
|
||||
+ "\"finish_reason\":\"tool_calls\"}],\"usage\":{\"prompt_tokens\":" + promptChars / 4 + ",\"completion_tokens\":12,\"total_tokens\":"
|
||||
+ (promptChars / 4 + 12) + "}}";
|
||||
}
|
||||
|
||||
private static String text(String text, int promptChars) {
|
||||
return "{\"id\":\"chatcmpl-x\",\"object\":\"chat.completion\",\"created\":1700000000,\"model\":\"gpt-6.1-sol\","
|
||||
+ "\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":" + JSON.writeValueAsString(text)
|
||||
+ "},\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":" + promptChars / 4 + ",\"completion_tokens\":"
|
||||
+ Math.max(1, text.length() / 4) + ",\"total_tokens\":" + (promptChars / 4 + Math.max(1, text.length() / 4)) + "}}";
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() { server.stop(0); }
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.List;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.ai.chat.client.ChatClient;
|
||||
import org.springframework.ai.mcp.SyncMcpToolCallbackProvider;
|
||||
import org.springframework.ai.mcp.annotation.McpTool;
|
||||
import org.springframework.ai.mcp.annotation.McpToolParam;
|
||||
import org.springframework.boot.WebApplicationType;
|
||||
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
|
||||
import org.springframework.boot.builder.SpringApplicationBuilder;
|
||||
import org.springframework.context.ConfigurableApplicationContext;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
|
||||
import dev.langchain4j.mcp.McpToolProvider;
|
||||
import dev.langchain4j.mcp.client.DefaultMcpClient;
|
||||
import dev.langchain4j.mcp.client.transport.http.StreamableHttpMcpTransport;
|
||||
import dev.langchain4j.service.AiServices;
|
||||
import io.modelcontextprotocol.client.McpClient;
|
||||
import io.modelcontextprotocol.client.transport.HttpClientStreamableHttpTransport;
|
||||
|
||||
/** The order tool written once, behind an MCP server, and used from both frameworks as a client. */
|
||||
class McpFirstTest {
|
||||
|
||||
/** Nothing here imports either framework's tool annotation: only MCP's. */
|
||||
public static class McpOrderTools {
|
||||
|
||||
private final OrderService orders;
|
||||
|
||||
public McpOrderTools(OrderService orders) {
|
||||
this.orders = orders;
|
||||
}
|
||||
|
||||
@McpTool(name = "getOrderStatus", description = "Get current order status by order ID")
|
||||
public String getOrderStatus(@McpToolParam(description = "The unique identifier of the order", required = true) String orderId) {
|
||||
return orders.describe(orderId);
|
||||
}
|
||||
}
|
||||
|
||||
@Configuration
|
||||
@EnableAutoConfiguration
|
||||
static class ServerApp {
|
||||
|
||||
static final OrderService ORDERS = new OrderService();
|
||||
|
||||
@Bean
|
||||
McpOrderTools mcpOrderTools() {
|
||||
return new McpOrderTools(ORDERS);
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void theToolBehindMcpServesBothFrameworks() throws Exception {
|
||||
try (FakeOpenAi fake = new FakeOpenAi();
|
||||
ConfigurableApplicationContext server = new SpringApplicationBuilder(ServerApp.class)
|
||||
.web(WebApplicationType.SERVLET)
|
||||
.properties("server.port=0", "spring.ai.openai.api-key=unused",
|
||||
"spring.ai.mcp.client.enabled=false",
|
||||
"spring.autoconfigure.exclude=dev.langchain4j.openai.spring.OpenAiAutoConfiguration",
|
||||
"spring.ai.mcp.server.name=orders", "spring.ai.mcp.server.version=1.0.0",
|
||||
"spring.ai.mcp.server.protocol=streamable")
|
||||
.run()) {
|
||||
int port = server.getEnvironment().getProperty("local.server.port", Integer.class);
|
||||
String base = "http://127.0.0.1:" + port;
|
||||
|
||||
// Spring AI as an MCP client: the tools arrive as ToolCallbacks.
|
||||
var sdkClient = McpClient.sync(HttpClientStreamableHttpTransport.builder(base).endpoint("/mcp").build())
|
||||
.requestTimeout(Duration.ofSeconds(10)).build();
|
||||
sdkClient.initialize();
|
||||
var springProvider = SyncMcpToolCallbackProvider.builder().mcpClients(sdkClient).build();
|
||||
ChatClient spring = ChatClient.builder(Wiring.springModel(fake.url())).defaultSystem(Wiring.SYSTEM)
|
||||
.defaultToolCallbacks(springProvider).build();
|
||||
String springAnswer = spring.prompt().user("Where is ORD-1002?").call().content();
|
||||
int springRequests = fake.bodies.size();
|
||||
String springTool = SameAssistantTest.summarise(fake.bodies.get(0)).stream().filter(l -> l.contains("tool getOrderStatus")).findFirst().orElse("(no tool offered)");
|
||||
int callsAfterSpring = ServerApp.ORDERS.calls.get();
|
||||
fake.reset();
|
||||
|
||||
// LangChain4j as an MCP client.
|
||||
var lcMcp = DefaultMcpClient.builder().key("orders")
|
||||
.transport(StreamableHttpMcpTransport.builder().url(base + "/mcp").build()).build();
|
||||
Support lc = AiServices.builder(Support.class).chatModel(Wiring.lc4jModel(fake.url()))
|
||||
.toolProvider(McpToolProvider.builder().mcpClients(List.of(lcMcp)).build()).build();
|
||||
String lcAnswer = lc.chat("Where is ORD-1002?");
|
||||
int lcRequests = fake.bodies.size();
|
||||
String lcTool = SameAssistantTest.summarise(fake.bodies.get(0)).stream().filter(l -> l.contains("tool getOrderStatus")).findFirst().orElse("(no tool offered)");
|
||||
int callsAfterBoth = ServerApp.ORDERS.calls.get();
|
||||
|
||||
try (var t = new Transcript("05-mcp-first.txt", "The order tool served once over MCP (Streamable HTTP), used from Spring AI and from LangChain4j")) {
|
||||
t.line("MCP server: Spring AI 2.0.1 spring-ai-starter-mcp-server-webmvc, one @McpTool method, on port %d.", port);
|
||||
t.line("Spring AI client : McpClient.sync(HttpClientStreamableHttpTransport) -> SyncMcpToolCallbackProvider -> ChatClient.defaultToolCallbacks");
|
||||
t.line("LangChain4j client: DefaultMcpClient + StreamableHttpMcpTransport -> McpToolProvider -> AiServices.toolProvider");
|
||||
t.blank();
|
||||
t.line("Spring AI answer : %s (%d requests to the model)", springAnswer, springRequests);
|
||||
t.line(" tool offered to the model: %s", springTool);
|
||||
t.line("LangChain4j answer : %s (%d requests to the model)", lcAnswer, lcRequests);
|
||||
t.line(" tool offered to the model: %s", lcTool);
|
||||
t.blank();
|
||||
t.line("Times the one tool method on the server ran: %d for Spring AI, then %d more for LangChain4j.", callsAfterSpring, callsAfterBoth - callsAfterSpring);
|
||||
}
|
||||
sdkClient.close();
|
||||
lcMcp.close();
|
||||
assertThat(springAnswer).contains("PROCESSING").isNotNull();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.TreeSet;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.ai.chat.client.ChatClient;
|
||||
import org.springframework.ai.openai.OpenAiChatModel;
|
||||
import org.springframework.ai.openai.OpenAiChatOptions;
|
||||
|
||||
import dev.langchain4j.micrometer.metrics.listeners.MicrometerMetricsChatModelListener;
|
||||
import dev.langchain4j.service.AiServices;
|
||||
import io.micrometer.core.instrument.Meter;
|
||||
import io.micrometer.core.instrument.MeterRegistry;
|
||||
import io.micrometer.core.instrument.simple.SimpleMeterRegistry;
|
||||
import io.micrometer.observation.ObservationRegistry;
|
||||
import io.micrometer.core.instrument.observation.DefaultMeterObservationHandler;
|
||||
|
||||
class ObservabilityTest {
|
||||
|
||||
static List<String> describe(MeterRegistry r) {
|
||||
return r.getMeters().stream().map(m -> {
|
||||
Meter.Id id = m.getId();
|
||||
String tags = id.getTags().stream().filter(t -> List.of("gen_ai.token.type", "gen_ai.request.model", "gen_ai.response.model",
|
||||
"gen_ai.operation.name", "gen_ai.provider.name", "spring.ai.kind", "model", "gen_ai.system", "tool.name", "error", "outcome")
|
||||
.contains(t.getKey())).map(t -> t.getKey() + "=" + t.getValue()).sorted().collect(Collectors.joining(", "));
|
||||
return id.getName() + " [" + id.getType() + "]" + (tags.isEmpty() ? "" : " " + tags);
|
||||
}).distinct().sorted().toList();
|
||||
}
|
||||
|
||||
@Test
|
||||
void whatEachFrameworkRecordsWithoutBeingAsked() throws Exception {
|
||||
try (FakeOpenAi fake = new FakeOpenAi()) {
|
||||
// Spring AI: a model and a client that share an ObservationRegistry; a handler turns observations into meters.
|
||||
SimpleMeterRegistry springMeters = new SimpleMeterRegistry();
|
||||
ObservationRegistry observations = ObservationRegistry.create();
|
||||
observations.observationConfig().observationHandler(new DefaultMeterObservationHandler(springMeters));
|
||||
// the handler Spring Boot registers for you; it turns the model's token usage into a meter
|
||||
observations.observationConfig().observationHandler(new org.springframework.ai.chat.observation.ChatModelMeterObservationHandler(springMeters));
|
||||
OrderService sOrders = new OrderService();
|
||||
OpenAiChatModel model = OpenAiChatModel.builder().observationRegistry(observations).options(OpenAiChatOptions.builder()
|
||||
.baseUrl(fake.url() + "/v1").apiKey("k").model("gpt-6.1-sol").maxRetries(0).build()).build();
|
||||
ChatClient spring = ChatClient.builder(model, observations, null, null).defaultSystem(Wiring.SYSTEM)
|
||||
.defaultTools(new SpringAiTools(sOrders)).build();
|
||||
spring.prompt().user("Where is ORD-1001?").call().content();
|
||||
|
||||
// LangChain4j: nothing is recorded unless a listener is registered on the model.
|
||||
SimpleMeterRegistry bare = new SimpleMeterRegistry();
|
||||
AiServices.builder(Support.class).chatModel(Wiring.lc4jModel(fake.url())).tools(new Lc4jTools(new OrderService())).build()
|
||||
.chat("Where is ORD-1001?");
|
||||
List<String> lcNone = describe(bare);
|
||||
|
||||
SimpleMeterRegistry lcOwn = new SimpleMeterRegistry(), lcShipped = new SimpleMeterRegistry();
|
||||
var withOwn = dev.langchain4j.model.openai.OpenAiChatModel.builder()
|
||||
.httpClientBuilder(dev.langchain4j.http.client.jdk.JdkHttpClient.builder())
|
||||
.baseUrl(fake.url() + "/v1").apiKey("k").modelName("gpt-6.1-sol")
|
||||
.listeners(List.of(new TracingChatInterceptor(lcOwn))).build();
|
||||
AiServices.builder(Support.class).chatModel(withOwn).tools(new Lc4jTools(new OrderService())).build().chat("Where is ORD-1001?");
|
||||
var withShipped = dev.langchain4j.model.openai.OpenAiChatModel.builder()
|
||||
.httpClientBuilder(dev.langchain4j.http.client.jdk.JdkHttpClient.builder())
|
||||
.baseUrl(fake.url() + "/v1").apiKey("k").modelName("gpt-6.1-sol")
|
||||
.listeners(List.of(new MicrometerMetricsChatModelListener(lcShipped))).build();
|
||||
AiServices.builder(Support.class).chatModel(withShipped).tools(new Lc4jTools(new OrderService())).build().chat("Where is ORD-1001?");
|
||||
|
||||
try (var t = new Transcript("04-observability.txt", "What each framework records for one question that uses one tool (2 model requests)")) {
|
||||
t.line("Spring AI 2.0.1: model and client given an ObservationRegistry plus DefaultMeterObservationHandler and ChatModelMeterObservationHandler (the two Spring Boot registers for you).");
|
||||
describe(springMeters).forEach(l -> t.line(" %s", l));
|
||||
t.blank();
|
||||
t.line("LangChain4j 1.22.0, no listener registered: %d meters.", lcNone.size());
|
||||
t.blank();
|
||||
t.line("LangChain4j 1.22.0 with TracingChatInterceptor, the listener from the original article (it still compiles and runs):");
|
||||
describe(lcOwn).forEach(l -> t.line(" %s", l));
|
||||
t.blank();
|
||||
t.line("LangChain4j 1.22.0 with the shipped MicrometerMetricsChatModelListener (langchain4j-micrometer-metrics 1.22.0-beta32):");
|
||||
describe(lcShipped).forEach(l -> t.line(" %s", l));
|
||||
}
|
||||
assertThat(springMeters.getMeters()).isNotEmpty();
|
||||
assertThat(lcNone).isEmpty();
|
||||
assertThat(lcOwn.getMeters()).isNotEmpty();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.TreeSet;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import tools.jackson.databind.JsonNode;
|
||||
import tools.jackson.databind.json.JsonMapper;
|
||||
|
||||
class SameAssistantTest {
|
||||
|
||||
static final JsonMapper JSON = JsonMapper.builder().build();
|
||||
|
||||
/** One request body reduced to what a reader compares: top-level fields, messages and tools. */
|
||||
static List<String> summarise(String body) {
|
||||
JsonNode req = JSON.readTree(body);
|
||||
List<String> out = new ArrayList<>();
|
||||
TreeSet<String> keys = new TreeSet<>();
|
||||
req.propertyNames().forEach(keys::add);
|
||||
out.add("fields: " + keys);
|
||||
for (JsonNode m : req.get("messages")) {
|
||||
String role = m.get("role").asText();
|
||||
String detail;
|
||||
if (m.has("tool_calls") && m.get("tool_calls").size() > 0) {
|
||||
JsonNode f = m.get("tool_calls").get(0).get("function");
|
||||
detail = "tool_call " + f.get("name").asText() + " " + f.get("arguments").asText();
|
||||
} else {
|
||||
JsonNode c = m.get("content");
|
||||
detail = c == null || c.isNull() ? "" : c.isString() ? "\"" + c.asString() + "\"" : c.toString();
|
||||
}
|
||||
out.add(" " + role + ": " + detail);
|
||||
}
|
||||
if (req.has("tools")) {
|
||||
for (JsonNode tool : req.get("tools")) {
|
||||
JsonNode f = tool.get("function");
|
||||
out.add(" tool " + f.get("name").asText() + " - " + f.path("description").asText("(no description)") + " params " + f.path("parameters"));
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
static String names(JsonNode n) {
|
||||
TreeSet<String> k = new TreeSet<>();
|
||||
n.propertyNames().forEach(k::add);
|
||||
return k.toString();
|
||||
}
|
||||
|
||||
@Test
|
||||
void theSameOrderAssistantInBothFrameworks() throws Exception {
|
||||
try (FakeOpenAi fake = new FakeOpenAi()) {
|
||||
OrderService springOrders = new OrderService(), lcOrders = new OrderService();
|
||||
String springAnswer = Wiring.springAssistant(fake.url(), springOrders).prompt().user("Where is ORD-1001?").call().content();
|
||||
List<String> springBodies = new ArrayList<>(fake.bodies);
|
||||
fake.reset();
|
||||
String lcAnswer = Wiring.lc4jAssistant(fake.url(), lcOrders).chat("Where is ORD-1001?");
|
||||
List<String> lcBodies = new ArrayList<>(fake.bodies);
|
||||
|
||||
try (var t = new Transcript("02-same-assistant.txt", "The same order assistant in Spring AI and in LangChain4j, one question: \"Where is ORD-1001?\"")) {
|
||||
t.line("Spring AI : ChatClient.builder(model).defaultSystem(..).defaultTools(new SpringAiTools(orders))");
|
||||
t.line("LangChain4j: AiServices.builder(Support.class).chatModel(model).tools(new Lc4jTools(orders))");
|
||||
t.blank();
|
||||
t.line("Spring AI answer : %s", springAnswer);
|
||||
t.line("LangChain4j answer : %s", lcAnswer);
|
||||
t.line("HTTP requests sent : Spring AI %d, LangChain4j %d. Tool body executed: Spring AI %d, LangChain4j %d.",
|
||||
springBodies.size(), lcBodies.size(), springOrders.calls.get(), lcOrders.calls.get());
|
||||
t.blank();
|
||||
t.line("What differs on the wire (computed from the four bodies below):");
|
||||
JsonNode s1 = JSON.readTree(springBodies.get(0)), l1 = JSON.readTree(lcBodies.get(0));
|
||||
JsonNode s2 = JSON.readTree(springBodies.get(1)), l2 = JSON.readTree(lcBodies.get(1));
|
||||
t.line(" top-level fields : Spring AI %s, LangChain4j %s", names(s1), names(l1));
|
||||
t.line(" tool schema : Spring AI has $schema=%s additionalProperties=%s; LangChain4j has $schema=%s additionalProperties=%s",
|
||||
s1.at("/tools/0/function/parameters").has("$schema"), s1.at("/tools/0/function/parameters").has("additionalProperties"),
|
||||
l1.at("/tools/0/function/parameters").has("$schema"), l1.at("/tools/0/function/parameters").has("additionalProperties"));
|
||||
t.line(" tool result message: Spring AI sent %s, LangChain4j sent %s", s2.at("/messages/3/content"), l2.at("/messages/3/content"));
|
||||
for (int i = 0; i < springBodies.size(); i++) {
|
||||
t.blank();
|
||||
t.line("--- Spring AI, request %d", i + 1);
|
||||
summarise(springBodies.get(i)).forEach(l -> t.line("%s", l));
|
||||
}
|
||||
for (int i = 0; i < lcBodies.size(); i++) {
|
||||
t.blank();
|
||||
t.line("--- LangChain4j, request %d", i + 1);
|
||||
summarise(lcBodies.get(i)).forEach(l -> t.line("%s", l));
|
||||
}
|
||||
}
|
||||
assertThat(springAnswer.replace("\"", "")).isEqualTo(lcAnswer);
|
||||
assertThat(springAnswer).isNotEqualTo(lcAnswer);
|
||||
assertThat(springBodies).hasSize(2);
|
||||
assertThat(lcBodies).hasSize(2);
|
||||
assertThat(springOrders.calls.get()).isEqualTo(1);
|
||||
assertThat(lcOrders.calls.get()).isEqualTo(1);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import java.util.function.Supplier;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import tools.jackson.databind.JsonNode;
|
||||
import tools.jackson.databind.json.JsonMapper;
|
||||
|
||||
class ToolFailureTest {
|
||||
|
||||
static final JsonMapper JSON = JsonMapper.builder().build();
|
||||
|
||||
record Run(String result, int requests, int toolBodies, String toolMessage) { }
|
||||
|
||||
static Run run(FakeOpenAi fake, OrderService orders, Supplier<String> call) {
|
||||
fake.bodies.clear();
|
||||
orders.calls.set(0);
|
||||
String result;
|
||||
try {
|
||||
result = "answer: " + call.get();
|
||||
} catch (Exception e) {
|
||||
Throwable root = e;
|
||||
while (root.getCause() != null) root = root.getCause();
|
||||
result = "exception " + e.getClass().getSimpleName() + (root != e ? " (root: " + root.getClass().getSimpleName() + ")" : "")
|
||||
+ ": " + String.valueOf(root.getMessage()).lines().findFirst().orElse("");
|
||||
}
|
||||
String toolMsg = "(none sent)";
|
||||
for (String b : fake.bodies) {
|
||||
JsonNode msgs = JSON.readTree(b).get("messages");
|
||||
for (JsonNode m : msgs) if (m.get("role").asText().equals("tool")) toolMsg = m.get("content").toString();
|
||||
}
|
||||
return new Run(result, fake.bodies.size(), orders.calls.get(), toolMsg);
|
||||
}
|
||||
|
||||
@Test
|
||||
void whatEachFrameworkDoesWhenAToolThrows() throws Exception {
|
||||
try (FakeOpenAi fake = new FakeOpenAi()) {
|
||||
OrderService sOrders = new OrderService(), lOrders = new OrderService();
|
||||
var spring = Wiring.springAssistant(fake.url(), sOrders);
|
||||
var lc = Wiring.lc4jAssistant(fake.url(), lOrders);
|
||||
|
||||
Run s1 = run(fake, sOrders, () -> spring.prompt().user("Where is ORD-500?").call().content());
|
||||
Run l1 = run(fake, lOrders, () -> lc.chat("Where is ORD-500?"));
|
||||
|
||||
// A model that keeps asking for the tool after every result, stopped by the server after 30 requests.
|
||||
fake.mode = FakeOpenAi.Mode.INSISTENT;
|
||||
fake.cutoff = 30;
|
||||
Run s2 = run(fake, sOrders, () -> spring.prompt().user("Where is ORD-500?").call().content());
|
||||
Run l2 = run(fake, lOrders, () -> lc.chat("Where is ORD-500?"));
|
||||
// The same insistent model, now with each framework's own way of capping the rounds at 5.
|
||||
fake.cutoff = 30;
|
||||
var cappedSpring = org.springframework.ai.chat.client.ChatClient.builder(Wiring.springModel(fake.url()))
|
||||
.defaultSystem(Wiring.SYSTEM).defaultTools(new SpringAiTools(sOrders))
|
||||
.defaultAdvisors(org.springframework.ai.chat.client.advisor.ToolCallingAdvisor.builder()
|
||||
.toolExecutionEligibilityChecker(new MaxToolRounds(5)).build()).build();
|
||||
var cappedLc = dev.langchain4j.service.AiServices.builder(Support.class).chatModel(Wiring.lc4jModel(fake.url()))
|
||||
.tools(new Lc4jTools(lOrders)).maxSequentialToolsInvocations(5).build();
|
||||
Run s3 = run(fake, sOrders, () -> String.valueOf(cappedSpring.prompt().user("Where is ORD-500?").call().content()));
|
||||
Run l3 = run(fake, lOrders, () -> cappedLc.chat("Where is ORD-500?"));
|
||||
fake.reset();
|
||||
|
||||
try (var t = new Transcript("03-tool-throws.txt", "A tool that throws (ORD-500 raises IllegalStateException: order database unavailable)")) {
|
||||
t.line("Case 1: a well-behaved model. It asks for the tool once, then answers from whatever the tool message says.");
|
||||
t.line(" Spring AI : %d requests, tool body ran %d time(s). Tool message the model received: %s", s1.requests(), s1.toolBodies(), s1.toolMessage());
|
||||
t.line(" %s", s1.result());
|
||||
t.line(" LangChain4j : %d requests, tool body ran %d time(s). Tool message the model received: %s", l1.requests(), l1.toolBodies(), l1.toolMessage());
|
||||
t.line(" %s", l1.result());
|
||||
t.blank();
|
||||
t.line("Case 2: an insistent model. It asks for the tool again after every result. The server stops it with an HTTP 500 after 30 requests.");
|
||||
t.line(" Spring AI : %d requests, tool body ran %d time(s)", s2.requests(), s2.toolBodies());
|
||||
t.line(" %s", s2.result());
|
||||
t.line(" LangChain4j : %d requests, tool body ran %d time(s)", l2.requests(), l2.toolBodies());
|
||||
t.line(" %s", l2.result());
|
||||
t.blank();
|
||||
t.line("Case 3: the same insistent model, each framework asked to stop after 5 tool rounds.");
|
||||
t.line(" Spring AI (ToolCallingAdvisor + our MaxToolRounds(5) checker): %d requests, tool body ran %d time(s)", s3.requests(), s3.toolBodies());
|
||||
t.line(" %s", s3.result());
|
||||
t.line(" LangChain4j (maxSequentialToolsInvocations(5)) : %d requests, tool body ran %d time(s)", l3.requests(), l3.toolBodies());
|
||||
t.line(" %s", l3.result());
|
||||
}
|
||||
assertThat(s1.requests()).isGreaterThan(0);
|
||||
assertThat(l1.requests()).isGreaterThan(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
package com.ankurm.interop;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.io.PrintWriter;
|
||||
import java.io.StringWriter;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
|
||||
/** Writes a numbered transcript under {@code output/} and echoes it. Every console block in the article is one of these files. */
|
||||
final class Transcript implements AutoCloseable {
|
||||
|
||||
private final Path path;
|
||||
private final StringWriter buffer = new StringWriter();
|
||||
private final PrintWriter out = new PrintWriter(buffer);
|
||||
|
||||
Transcript(String fileName, String title) {
|
||||
this.path = Path.of("output", fileName);
|
||||
out.println("# " + title);
|
||||
out.println();
|
||||
}
|
||||
|
||||
Transcript line(String format, Object... args) {
|
||||
out.println(args.length == 0 ? format : String.format(format, args));
|
||||
return this;
|
||||
}
|
||||
|
||||
Transcript blank() {
|
||||
out.println();
|
||||
return this;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
out.flush();
|
||||
try {
|
||||
Files.createDirectories(path.getParent());
|
||||
Files.writeString(path, buffer.toString());
|
||||
} catch (IOException e) {
|
||||
throw new IllegalStateException("could not write " + path, e);
|
||||
}
|
||||
System.out.print(buffer);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user