Add getting-started module: ChatClient fluent API, system/user prompts, streaming, and provider switching via spring.ai.model.chat
This commit is contained in:
@@ -4,6 +4,7 @@ Runnable companion code for the Spring AI articles on [ankurm.com](https://ankur
|
||||
|
||||
| Module | What it is | Article |
|
||||
|---|---|---|
|
||||
| [`getting-started/`](getting-started) | One `ChatClient` bean, three endpoints (plain call, templated system prompt, streaming), and a test proving `spring.ai.model.chat` switches providers with zero code change. Spring Boot 4.1.1, Spring AI 2.0.1, Java 25. | [Spring AI 2.0 in 10 Minutes: ChatClient on Spring Boot 4.1](https://ankurm.com/spring-ai-2-0-chatclient-boot-4-1/) |
|
||||
| [`rag/`](rag) | Ingest PDFs, chunk, retrieve from pgvector, rerank, answer, check the answer. Spring Boot 4.1.1, Spring AI 2.0.1, Java 25. | [Production-grade RAG with Spring AI](https://ankurm.com/production-rag-spring-ai-java/) and [the complete example](https://ankurm.com/spring-ai-rag-complete-example/) |
|
||||
|
||||
Upgrading from Spring AI 1.x: [migration guide](https://ankurm.com/spring-ai-1-to-2-migration-guide/).
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
target/
|
||||
@@ -0,0 +1,58 @@
|
||||
# getting-started
|
||||
|
||||
Companion code for [Spring AI 2.0 in 10 Minutes: ChatClient on Spring Boot 4.1](https://ankurm.com/spring-ai-2-0-chatclient-boot-4-1/)
|
||||
on [ankurm.com](https://ankurm.com). One `ChatClient` bean, three endpoints, and a test that
|
||||
proves `spring.ai.model.chat` switches the provider without touching either.
|
||||
|
||||
The deeper walkthrough (what each test caught, the API references) lives in the article's
|
||||
accordion sections, not in this README -- see the versions callout and the "going deeper" list
|
||||
at the end of each section.
|
||||
|
||||
## Versions
|
||||
|
||||
| Component | Version |
|
||||
|---|---|
|
||||
| Spring Boot | 4.1.1 |
|
||||
| Spring AI | 2.0.1 |
|
||||
| Java | 25 (LTS) |
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
mvn -o test # runs against a scripted ChatModel, no key needed, ~10s
|
||||
OPENAI_API_KEY=sk-... ./scripts/run.sh # the real app, port 8080
|
||||
```
|
||||
|
||||
## Endpoints
|
||||
|
||||
| Method | Path | What it shows |
|
||||
|---|---|---|
|
||||
| GET | `/api/chat?message=` | `ChatClient.prompt().user(...).call().content()` |
|
||||
| GET | `/api/chat/as?voice=&message=` | a templated system prompt filled via `.param(...)` |
|
||||
| GET | `/api/chat/stream?message=` | `.stream().content()`, a `Flux<String>` |
|
||||
|
||||
## Switching providers
|
||||
|
||||
One property, no code change:
|
||||
|
||||
```properties
|
||||
spring.ai.model.chat=openai # org.springframework.ai.openai.OpenAiChatModel
|
||||
spring.ai.model.chat=ollama # org.springframework.ai.ollama.OllamaChatModel
|
||||
```
|
||||
|
||||
`ProviderSwitchTest` proves this mechanically -- it never calls either provider's server, only
|
||||
inspects which `ChatModel` bean class the Spring context assembled under each property value.
|
||||
|
||||
## Output
|
||||
|
||||
| File | Regenerated by |
|
||||
|---|---|
|
||||
| `output/01-plain-call.txt` | `ChatClientEndpointsTest` |
|
||||
| `output/02-system-template.txt` | `ChatClientEndpointsTest` |
|
||||
| `output/03-streaming.txt` | `ChatClientEndpointsTest` |
|
||||
| `output/04-provider-switch.txt` | `ProviderSwitchTest` |
|
||||
| `output/05-real-network-round-trip.txt` | captured manually against the real OpenAI API; see its own header |
|
||||
|
||||
`scripts/run-all.sh` regenerates 01-04. 05 is not part of that script -- it needs a live network
|
||||
path and a real (even if invalid) key, and is included once as evidence that the wiring reaches
|
||||
OpenAI's own server.
|
||||
@@ -0,0 +1,5 @@
|
||||
# GET /api/chat -- ChatClient.prompt().user(...).call().content()
|
||||
|
||||
$ curl 'http://localhost:34201/api/chat?message=What+package+is+ChatClient+in?'
|
||||
|
||||
You said: "What package is ChatClient in?". That is 30 characters.
|
||||
@@ -0,0 +1,9 @@
|
||||
# GET /api/chat/as -- a templated system prompt, filled via .param("voice", ...)
|
||||
|
||||
$ curl 'http://localhost:34201/api/chat/as?voice=pirate&message=Where+is+my+jar+cached'
|
||||
|
||||
Response body:
|
||||
As a pirate: Where is my jar cached -- yes, and it is exactly 22 characters long.
|
||||
|
||||
Exact text FakeChatModel received (proves the {voice} placeholder was substituted by ChatClient before the ChatModel ever saw the prompt):
|
||||
Answer in the voice of a pirate, still in two sentences.Where is my jar cached
|
||||
@@ -0,0 +1,6 @@
|
||||
# GET /api/chat/stream -- ChatClient.prompt().user(...).stream().content()
|
||||
|
||||
$ curl -N 'http://localhost:34201/api/chat/stream?message=Stream+this'
|
||||
|
||||
Chunks received: 1
|
||||
Joined: You said: "Stream this". That is 11 characters.
|
||||
@@ -0,0 +1,4 @@
|
||||
# Which ChatModel class spring.ai.model.chat selects
|
||||
|
||||
spring.ai.model.chat=openai -> org.springframework.ai.openai.OpenAiChatModel
|
||||
spring.ai.model.chat=ollama -> org.springframework.ai.ollama.OllamaChatModel
|
||||
@@ -0,0 +1,26 @@
|
||||
# A real HTTP round trip to OpenAI (not scripted) -- captured manually, not by run-all.sh
|
||||
|
||||
This one is not reproducible by scripts/run-all.sh: it needs a live network path to
|
||||
api.openai.com, which the automated test suite deliberately does not depend on (see
|
||||
ChatClientEndpointsTest, which uses FakeChatModel instead). It is included because it is the
|
||||
one exhibit in this module that touches a real provider, and it proves the request pipeline
|
||||
-- ChatClient, the OpenAiChatModel bean spring.ai.model.chat=openai selected, and the OpenAI
|
||||
Java SDK underneath it -- is wired correctly all the way to OpenAI's own server, not just to
|
||||
a stub.
|
||||
|
||||
Command (full application, spring-boot:run, a syntactically valid but fake key):
|
||||
|
||||
$ export OPENAI_API_KEY=sk-test-placeholder-not-a-real-key
|
||||
$ curl 'http://localhost:8080/api/chat?message=hi'
|
||||
|
||||
Response: HTTP 500, body in 05-real-openai-401-response-body.txt
|
||||
|
||||
Application log root cause (grepped from the running process's stdout):
|
||||
|
||||
com.openai.errors.UnauthorizedException: 401: Incorrect API key provided: sk-test-**********************-key. You can find your API key at https://platform.openai.com/account/api-keys.
|
||||
|
||||
That 401 comes from OpenAI's own server, not from Spring AI or from this sandbox -- OpenAI
|
||||
validated the key format, rejected it, and said so in its own error class
|
||||
(com.openai.errors.UnauthorizedException). A malformed request, a wrong base URL, or a client
|
||||
that never left the JVM would not look like this. Swap in a real key and this becomes a real
|
||||
reply.
|
||||
@@ -0,0 +1 @@
|
||||
{"timestamp":"2026-09-23T03:53:37.888Z","status":500,"error":"Internal Server Error","path":"/api/chat"}
|
||||
@@ -0,0 +1,12 @@
|
||||
# The original surefire failure that led to overriding FakeChatModel.stream()
|
||||
|
||||
Captured from `mvn -B -o test` before FakeChatModel had a stream() override -- ChatModel's own
|
||||
default implementation was assumed (wrongly) to wrap call() in a single-element Flux. Trimmed to
|
||||
the relevant frames; jar version suffixes kept exactly as printed.
|
||||
|
||||
2026-09-23T03:50:42.884Z ERROR 1145 --- [getting-started] [o-auto-1-exec-3] o.s.ai.chat.model.MessageAggregator : Aggregation Error
|
||||
|
||||
java.lang.UnsupportedOperationException: streaming is not supported
|
||||
at org.springframework.ai.chat.model.ChatModel.stream(ChatModel.java:65) ~[spring-ai-model-2.0.1.jar:2.0.1]
|
||||
at org.springframework.ai.chat.client.advisor.ChatModelStreamAdvisor.adviseStream(ChatModelStreamAdvisor.java:53) ~[spring-ai-client-chat-2.0.1.jar:2.0.1]
|
||||
at org.springframework.ai.chat.client.advisor.DefaultAroundAdvisorChain.lambda$nextStream$5(DefaultAroundAdvisorChain.java:158) ~[spring-ai-client-chat-2.0.1.jar:2.0.1]
|
||||
@@ -0,0 +1,85 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-parent</artifactId>
|
||||
<version>4.1.1</version>
|
||||
<relativePath/>
|
||||
</parent>
|
||||
|
||||
<groupId>com.ankurm</groupId>
|
||||
<artifactId>getting-started</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<name>getting-started</name>
|
||||
<description>ChatClient on Spring Boot 4.1 and Spring AI 2.0: the fluent API, system/user prompts, streaming, and switching chat providers by property</description>
|
||||
|
||||
<properties>
|
||||
<java.version>25</java.version>
|
||||
<!-- Spring AI is not managed by the Spring Boot BOM: this pair is yours to keep compatible. -->
|
||||
<spring-ai.version>2.0.1</spring-ai.version>
|
||||
</properties>
|
||||
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-bom</artifactId>
|
||||
<version>${spring-ai.version}</version>
|
||||
<type>pom</type>
|
||||
<scope>import</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</dependencyManagement>
|
||||
|
||||
<dependencies>
|
||||
<!-- spring-boot-starter-web is deprecated in Boot 4; the WebMVC starter is now named for the
|
||||
stack it actually brings, since spring-boot-starter-webflux is its sibling. -->
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-webmvc</artifactId>
|
||||
</dependency>
|
||||
|
||||
<!-- Chat from OpenAI. The starter was spring-ai-openai-spring-boot-starter in 1.x. -->
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-starter-model-openai</artifactId>
|
||||
</dependency>
|
||||
<!-- A second provider on the classpath at once, purely to prove that spring.ai.model.chat
|
||||
picks between them at runtime with no code change. Its own server is never contacted here. -->
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-starter-model-ollama</artifactId>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-test</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>io.projectreactor</groupId>
|
||||
<artifactId>reactor-test</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-surefire-plugin</artifactId>
|
||||
<configuration>
|
||||
<argLine>-Duser.timezone=UTC -Dstdout.encoding=UTF-8 -Dfile.encoding=UTF-8</argLine>
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
Executable
+13
@@ -0,0 +1,13 @@
|
||||
#!/usr/bin/env bash
|
||||
# Regenerates every reproducible file under output/. Run from the module root
|
||||
# (spring-ai/getting-started). Needs Maven with the dependencies already resolved once online.
|
||||
#
|
||||
# output/05-*.txt is NOT regenerated here -- it needs a live network path to api.openai.com and a
|
||||
# running instance of the app started separately. See its own header for the exact commands.
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
mvn -q -o test
|
||||
echo
|
||||
echo "Regenerated output/01-plain-call.txt, 02-system-template.txt, 03-streaming.txt, 04-provider-switch.txt"
|
||||
echo "output/05-real-network-round-trip.txt was captured manually -- see its header."
|
||||
Executable
+19
@@ -0,0 +1,19 @@
|
||||
#!/usr/bin/env bash
|
||||
# Starts the application, killing any previous instance first.
|
||||
#
|
||||
# Usage:
|
||||
# OPENAI_API_KEY=sk-... ./scripts/run.sh
|
||||
# CHAT_PROVIDER=ollama OLLAMA_BASE_URL=http://localhost:11434 ./scripts/run.sh
|
||||
#
|
||||
# Then, in another shell:
|
||||
# curl 'http://localhost:8080/api/chat?message=What+is+a+ChatClient?'
|
||||
# curl 'http://localhost:8080/api/chat/as?voice=pirate&message=Where+is+my+jar+cached'
|
||||
# curl -N 'http://localhost:8080/api/chat/stream?message=Stream+this'
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
for p in $(ps -eo pid,cmd | grep '[G]ettingStartedApplication' | awk '{print $1}'); do
|
||||
kill -9 "$p"
|
||||
done
|
||||
|
||||
mvn -q -o org.springframework.boot:spring-boot-maven-plugin:run
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
package com.ankurm.gettingstarted;
|
||||
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
|
||||
@SpringBootApplication
|
||||
public class GettingStartedApplication {
|
||||
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(GettingStartedApplication.class, args);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
package com.ankurm.gettingstarted.config;
|
||||
|
||||
import org.springframework.ai.chat.client.ChatClient;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
|
||||
/**
|
||||
* The one bean this module exists to explain. {@link ChatClient.Builder} arrives
|
||||
* autoconfigured and already pointed at whichever {@code ChatModel} bean
|
||||
* {@code spring.ai.model.chat} selected — this class never mentions OpenAI or Ollama
|
||||
* by name, and that is the whole point of the properties-based switch.
|
||||
*/
|
||||
@Configuration
|
||||
public class ChatClientConfig {
|
||||
|
||||
@Bean
|
||||
ChatClient chatClient(ChatClient.Builder builder) {
|
||||
return builder
|
||||
.defaultSystem("""
|
||||
You are a terse Java and Spring assistant. Answer in at most two \
|
||||
sentences unless the user asks for code.""")
|
||||
.build();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
package com.ankurm.gettingstarted.web;
|
||||
|
||||
import org.springframework.ai.chat.client.ChatClient;
|
||||
import org.springframework.http.MediaType;
|
||||
import org.springframework.web.bind.annotation.GetMapping;
|
||||
import org.springframework.web.bind.annotation.RequestParam;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
import reactor.core.publisher.Flux;
|
||||
|
||||
/**
|
||||
* Three endpoints, three corners of the {@link ChatClient} fluent API:
|
||||
* a plain call, a call with a templated system prompt, and a streamed call.
|
||||
* See {@code docs} on ankurm.com for the walkthrough — the endpoints
|
||||
* themselves carry no comment beyond what the method name says, deliberately,
|
||||
* since the interesting part is the one-line body.
|
||||
*/
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final ChatClient chatClient;
|
||||
|
||||
public ChatController(ChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
}
|
||||
|
||||
@GetMapping("/api/chat")
|
||||
public String chat(@RequestParam String message) {
|
||||
return this.chatClient.prompt()
|
||||
.user(message)
|
||||
.call()
|
||||
.content();
|
||||
}
|
||||
|
||||
@GetMapping("/api/chat/as")
|
||||
public String chatAs(@RequestParam String voice, @RequestParam String message) {
|
||||
return this.chatClient.prompt()
|
||||
.system(s -> s.text("Answer in the voice of a {voice}, still in two sentences.")
|
||||
.param("voice", voice))
|
||||
.user(message)
|
||||
.call()
|
||||
.content();
|
||||
}
|
||||
|
||||
@GetMapping(value = "/api/chat/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
|
||||
public Flux<String> chatStream(@RequestParam String message) {
|
||||
return this.chatClient.prompt()
|
||||
.user(message)
|
||||
.stream()
|
||||
.content();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
spring:
|
||||
application:
|
||||
name: getting-started
|
||||
ai:
|
||||
# Which provider's ChatModel bean gets built. Flip this one line, keep the code
|
||||
# in ChatClientConfig and ChatController unchanged, and add the matching provider
|
||||
# properties below.
|
||||
model:
|
||||
chat: ${CHAT_PROVIDER:openai}
|
||||
openai:
|
||||
api-key: ${OPENAI_API_KEY:}
|
||||
chat:
|
||||
model: gpt-4o
|
||||
temperature: 0.1
|
||||
ollama:
|
||||
base-url: ${OLLAMA_BASE_URL:http://localhost:11434}
|
||||
chat:
|
||||
model: llama3.2
|
||||
@@ -0,0 +1,121 @@
|
||||
package com.ankurm.gettingstarted;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.List;
|
||||
|
||||
import com.ankurm.gettingstarted.config.TestChatModelConfig;
|
||||
import com.ankurm.gettingstarted.support.FakeChatModel;
|
||||
import com.ankurm.gettingstarted.support.Transcript;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.boot.test.web.server.LocalServerPort;
|
||||
import org.springframework.test.context.ActiveProfiles;
|
||||
import org.springframework.test.context.TestPropertySource;
|
||||
import org.springframework.test.web.reactive.server.WebTestClient;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
|
||||
@TestPropertySource(properties = "spring.ai.model.chat=none")
|
||||
@org.springframework.context.annotation.Import(TestChatModelConfig.class)
|
||||
class ChatClientEndpointsTest {
|
||||
|
||||
@LocalServerPort
|
||||
int port;
|
||||
|
||||
@Autowired
|
||||
FakeChatModel fakeChatModel;
|
||||
|
||||
private WebTestClient client() {
|
||||
return WebTestClient.bindToServer()
|
||||
.baseUrl("http://localhost:" + port)
|
||||
.responseTimeout(Duration.ofSeconds(10))
|
||||
.build();
|
||||
}
|
||||
|
||||
@Test
|
||||
void plainCallReturnsChatClientContent() {
|
||||
try (Transcript t = new Transcript("01-plain-call.txt",
|
||||
"GET /api/chat -- ChatClient.prompt().user(...).call().content()")) {
|
||||
|
||||
String body = client().get()
|
||||
.uri("/api/chat?message=" + "What package is ChatClient in?")
|
||||
.exchange()
|
||||
.expectStatus().isOk()
|
||||
.expectBody(String.class)
|
||||
.returnResult()
|
||||
.getResponseBody();
|
||||
|
||||
t.line("$ curl 'http://localhost:%d/api/chat?message=What+package+is+ChatClient+in?'", port)
|
||||
.blank()
|
||||
.line(body);
|
||||
|
||||
assertThat(body).isEqualTo(
|
||||
"You said: \"What package is ChatClient in?\". That is 30 characters.");
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void systemPromptTemplateIsFilledBeforeItReachesTheModel() {
|
||||
try (Transcript t = new Transcript("02-system-template.txt",
|
||||
"GET /api/chat/as -- a templated system prompt, filled via .param(\"voice\", ...)")) {
|
||||
|
||||
String body = client().get()
|
||||
.uri("/api/chat/as?voice=pirate&message=Where+is+my+jar+cached")
|
||||
.exchange()
|
||||
.expectStatus().isOk()
|
||||
.expectBody(String.class)
|
||||
.returnResult()
|
||||
.getResponseBody();
|
||||
|
||||
List<String> sent = fakeChatModel.promptTexts();
|
||||
String lastPrompt = sent.get(sent.size() - 1);
|
||||
|
||||
t.line("$ curl 'http://localhost:%d/api/chat/as?voice=pirate&message=Where+is+my+jar+cached'", port)
|
||||
.blank()
|
||||
.line("Response body:")
|
||||
.line(body)
|
||||
.blank()
|
||||
.line("Exact text FakeChatModel received (proves the {voice} placeholder was substituted"
|
||||
+ " by ChatClient before the ChatModel ever saw the prompt):")
|
||||
.line(lastPrompt);
|
||||
|
||||
assertThat(lastPrompt)
|
||||
.contains("in the voice of a pirate")
|
||||
.doesNotContain("{voice}");
|
||||
assertThat(body).startsWith("As a pirate:");
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void streamEndpointReturnsAFluxOfContentChunks() {
|
||||
try (Transcript t = new Transcript("03-streaming.txt",
|
||||
"GET /api/chat/stream -- ChatClient.prompt().user(...).stream().content()")) {
|
||||
|
||||
List<String> chunks = client().get()
|
||||
.uri("/api/chat/stream?message=Stream+this")
|
||||
.accept(org.springframework.http.MediaType.TEXT_EVENT_STREAM)
|
||||
.exchange()
|
||||
.expectStatus().isOk()
|
||||
.returnResult(String.class)
|
||||
.getResponseBody()
|
||||
.collectList()
|
||||
.block(Duration.ofSeconds(10));
|
||||
|
||||
t.line("$ curl -N 'http://localhost:%d/api/chat/stream?message=Stream+this'", port)
|
||||
.blank()
|
||||
.line("Chunks received: %d", chunks.size())
|
||||
.line("Joined: %s", String.join("", chunks));
|
||||
|
||||
// FakeChatModel does not override stream(): ChatModel's own default implementation
|
||||
// wraps call() in a single-element Flux, so a stub model streams in exactly one
|
||||
// chunk. A real provider chunks token by token -- see the "going deeper" link below
|
||||
// for where that default is defined.
|
||||
assertThat(chunks).hasSize(1);
|
||||
assertThat(chunks.get(0)).isEqualTo(
|
||||
"You said: \"Stream this\". That is 11 characters.");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
package com.ankurm.gettingstarted;
|
||||
|
||||
import com.ankurm.gettingstarted.support.Transcript;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import org.springframework.ai.chat.model.ChatModel;
|
||||
import org.springframework.ai.model.ollama.autoconfigure.OllamaApiAutoConfiguration;
|
||||
import org.springframework.ai.model.ollama.autoconfigure.OllamaChatAutoConfiguration;
|
||||
import org.springframework.ai.model.openai.autoconfigure.OpenAiChatAutoConfiguration;
|
||||
import org.springframework.ai.model.tool.autoconfigure.ToolCallingAutoConfiguration;
|
||||
import org.springframework.boot.autoconfigure.AutoConfigurations;
|
||||
import org.springframework.boot.test.context.runner.ApplicationContextRunner;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
/**
|
||||
* Proves the claim the post makes in prose: flipping {@code spring.ai.model.chat} is enough to
|
||||
* change which {@code ChatModel} implementation Spring wires into {@code ChatClient.Builder},
|
||||
* with zero code change in {@code ChatClientConfig} or {@code ChatController}. Neither provider's
|
||||
* server is contacted -- this only inspects which bean class the context assembled.
|
||||
*
|
||||
* <p>{@code ToolCallingAutoConfiguration} has to be included too: {@code OpenAiChatAutoConfiguration}
|
||||
* autowires a {@code ToolCallingManager} constructor argument, and {@code ApplicationContextRunner}
|
||||
* only activates the autoconfigurations it is explicitly given, unlike a real Boot application
|
||||
* where every {@code AutoConfiguration.imports} entry on the classpath is a candidate. The first
|
||||
* run of this test failed context startup with {@code NoSuchBeanDefinitionException} for exactly
|
||||
* that type until this was added. {@code OllamaChatAutoConfiguration} needed the same fix for a
|
||||
* different reason: it depends on an {@code OllamaApi} bean, which its sibling
|
||||
* {@code OllamaApiAutoConfiguration} provides -- a real Boot app pulls both in automatically from
|
||||
* the same starter jar, but this runner has to be told about each one explicitly.
|
||||
*/
|
||||
class ProviderSwitchTest {
|
||||
|
||||
private final ApplicationContextRunner runner = new ApplicationContextRunner()
|
||||
.withConfiguration(AutoConfigurations.of(
|
||||
ToolCallingAutoConfiguration.class,
|
||||
OpenAiChatAutoConfiguration.class,
|
||||
OllamaApiAutoConfiguration.class,
|
||||
OllamaChatAutoConfiguration.class));
|
||||
|
||||
@Test
|
||||
void openaiPropertySelectsTheOpenAiChatModel() {
|
||||
try (Transcript t = new Transcript("04-provider-switch.txt",
|
||||
"Which ChatModel class spring.ai.model.chat selects")) {
|
||||
|
||||
runner.withPropertyValues(
|
||||
"spring.ai.model.chat=openai",
|
||||
"spring.ai.openai.api-key=sk-test-placeholder-never-sent")
|
||||
.run(context -> {
|
||||
assertThat(context).hasNotFailed();
|
||||
ChatModel model = context.getBean(ChatModel.class);
|
||||
t.line("spring.ai.model.chat=openai -> %s", model.getClass().getName());
|
||||
assertThat(model.getClass().getSimpleName()).isEqualTo("OpenAiChatModel");
|
||||
});
|
||||
|
||||
runner.withPropertyValues("spring.ai.model.chat=ollama")
|
||||
.run(context -> {
|
||||
ChatModel model = context.getBean(ChatModel.class);
|
||||
t.line("spring.ai.model.chat=ollama -> %s", model.getClass().getName());
|
||||
assertThat(model.getClass().getSimpleName()).isEqualTo("OllamaChatModel");
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
package com.ankurm.gettingstarted.config;
|
||||
|
||||
import com.ankurm.gettingstarted.support.FakeChatModel;
|
||||
import org.springframework.ai.chat.model.ChatModel;
|
||||
import org.springframework.boot.test.context.TestConfiguration;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Primary;
|
||||
|
||||
/**
|
||||
* Active only in tests that import it. Paired with {@code spring.ai.model.chat=none} on the test
|
||||
* itself, which stops the real OpenAI/Ollama autoconfiguration from building a {@code ChatModel}
|
||||
* at all, so this bean is the only one in the context.
|
||||
*/
|
||||
@TestConfiguration
|
||||
public class TestChatModelConfig {
|
||||
|
||||
@Bean
|
||||
@Primary
|
||||
FakeChatModel fakeChatModel() {
|
||||
return new FakeChatModel();
|
||||
}
|
||||
|
||||
@Bean
|
||||
ChatModel chatModel(FakeChatModel fake) {
|
||||
return fake;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
package com.ankurm.gettingstarted.support;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collections;
|
||||
import java.util.List;
|
||||
import java.util.concurrent.CopyOnWriteArrayList;
|
||||
|
||||
import org.springframework.ai.chat.messages.AssistantMessage;
|
||||
import org.springframework.ai.chat.model.ChatModel;
|
||||
import org.springframework.ai.chat.model.ChatResponse;
|
||||
import org.springframework.ai.chat.model.Generation;
|
||||
import org.springframework.ai.chat.prompt.Prompt;
|
||||
import reactor.core.publisher.Flux;
|
||||
|
||||
/**
|
||||
* A scripted {@link ChatModel}. It never leaves the JVM, so every test that wires it in is
|
||||
* a real Spring context wiring a real {@code ChatClient} around a stand-in for the network call
|
||||
* — what the tests in this module prove is that ChatClient assembled the prompt correctly,
|
||||
* never what a real OpenAI or Ollama reply would say.
|
||||
*
|
||||
* <p>The default-system prompt from {@code ChatClientConfig} and the voice-template prompt from
|
||||
* {@code ChatController#chatAs} are both text this module writes itself, so recognising them by a
|
||||
* fixed substring is matching against our own code, not guessing at a model's behaviour.
|
||||
*/
|
||||
public class FakeChatModel implements ChatModel {
|
||||
|
||||
private static final String DEFAULT_SYSTEM_MARKER = "terse Java and Spring assistant";
|
||||
private static final String DEFAULT_SYSTEM_SUFFIX = "unless the user asks for code.";
|
||||
private static final String VOICE_MARKER = "in the voice of a ";
|
||||
private static final String VOICE_SYSTEM_SUFFIX = "still in two sentences.";
|
||||
|
||||
private final List<Prompt> prompts = new CopyOnWriteArrayList<>();
|
||||
|
||||
public List<Prompt> prompts() {
|
||||
return Collections.unmodifiableList(prompts);
|
||||
}
|
||||
|
||||
@Override
|
||||
public ChatResponse call(Prompt prompt) {
|
||||
prompts.add(prompt);
|
||||
String contents = prompt.getContents();
|
||||
return new ChatResponse(List.of(new Generation(new AssistantMessage(reply(contents)))));
|
||||
}
|
||||
|
||||
/**
|
||||
* {@code ChatModel} does not implement {@code stream()} by delegating to {@code call()} --
|
||||
* its default throws {@code UnsupportedOperationException("streaming is not supported")}
|
||||
* (verified from the {@code ChatModel.java:65} stack frame the first run of this test
|
||||
* produced). A real provider streams the reply token by token; this stand-in streams it as
|
||||
* one chunk, which is honest about what it is standing in for.
|
||||
*/
|
||||
@Override
|
||||
public Flux<ChatResponse> stream(Prompt prompt) {
|
||||
return Flux.just(call(prompt));
|
||||
}
|
||||
|
||||
private static String reply(String contents) {
|
||||
int voiceIdx = contents.indexOf(VOICE_MARKER);
|
||||
if (voiceIdx >= 0) {
|
||||
int start = voiceIdx + VOICE_MARKER.length();
|
||||
int end = contents.indexOf(',', start);
|
||||
String voice = contents.substring(start, end < 0 ? contents.length() : end).strip();
|
||||
String user = afterSuffix(contents, VOICE_SYSTEM_SUFFIX);
|
||||
return "As a " + voice + ": " + user + " -- yes, and it is exactly " + user.length()
|
||||
+ " characters long.";
|
||||
}
|
||||
if (contents.contains(DEFAULT_SYSTEM_MARKER)) {
|
||||
String user = afterSuffix(contents, DEFAULT_SYSTEM_SUFFIX);
|
||||
return "You said: \"" + user + "\". That is " + user.length() + " characters.";
|
||||
}
|
||||
return "FakeChatModel saw a prompt it does not recognise: " + contents;
|
||||
}
|
||||
|
||||
/**
|
||||
* {@code Prompt#getContents()} joins every message's text with no separator at all -- not a
|
||||
* space, not a newline (verified: the first run of this test showed
|
||||
* {@code "...code.What package..."} glued together with no boundary). So the user text is
|
||||
* simply whatever follows this module's own fixed system-prompt suffix.
|
||||
*/
|
||||
private static String afterSuffix(String contents, String suffix) {
|
||||
int idx = contents.indexOf(suffix);
|
||||
return (idx >= 0 ? contents.substring(idx + suffix.length()) : contents).strip();
|
||||
}
|
||||
|
||||
/** Convenience for tests: every prompt's full joined text, for assertions on what was sent. */
|
||||
public List<String> promptTexts() {
|
||||
List<String> out = new ArrayList<>();
|
||||
for (Prompt p : prompts) {
|
||||
out.add(p.getContents());
|
||||
}
|
||||
return out;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
package com.ankurm.gettingstarted.support;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.io.PrintWriter;
|
||||
import java.io.StringWriter;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
|
||||
/**
|
||||
* Writes a numbered transcript under {@code output/} (repository root, not {@code docs/}) and
|
||||
* echoes it to the console. Every console block quoted in the article comes out of one of these
|
||||
* files verbatim.
|
||||
*/
|
||||
public final class Transcript implements AutoCloseable {
|
||||
|
||||
private final Path path;
|
||||
private final StringWriter buffer = new StringWriter();
|
||||
private final PrintWriter out = new PrintWriter(buffer);
|
||||
|
||||
public Transcript(String fileName, String title) {
|
||||
this.path = Path.of("output", fileName);
|
||||
out.println("# " + title);
|
||||
out.println();
|
||||
}
|
||||
|
||||
public Transcript line(String format, Object... args) {
|
||||
out.println(args.length == 0 ? format : String.format(format, args));
|
||||
return this;
|
||||
}
|
||||
|
||||
public Transcript blank() {
|
||||
out.println();
|
||||
return this;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
out.flush();
|
||||
try {
|
||||
Files.createDirectories(path.getParent());
|
||||
Files.writeString(path, buffer.toString());
|
||||
} catch (IOException e) {
|
||||
throw new IllegalStateException("could not write " + path, e);
|
||||
}
|
||||
System.out.print(buffer);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user