asmhatre/spring-ai
1
0
Fork 0
Code Actions Activity

Add ollama-local module: chat and embeddings against a real local Ollama server

Browse Source 
- spring-ai-starter-model-ollama autoconfigures ChatModel/EmbeddingModel from
  spring.ai.ollama.* properties alone; no API key anywhere in this module.
- org.testcontainers:ollama and org.testcontainers:junit-jupiter were both
  renamed in the Testcontainers 2.x line -- to org.testcontainers:testcontainers-ollama
  and org.testcontainers:testcontainers-junit-jupiter respectively -- confirmed by
  reading the real testcontainers-bom-2.0.5.pom that Spring Boot 4.1.1 imports
  (spring-boot-dependencies -> testcontainers.version=2.0.5). The pre-rename
  artifact IDs still exist on Maven Central but are stuck on the 1.x line.
- Unlike every other module in this series, tests drive a real local model
  (qwen2.5:0.5b chat, all-minilm embeddings) via a Testcontainers-managed
  OllamaContainer started from a baked image (scripts/bake-image.sh), not a
  ScriptedChatModel -- the whole point of this post is a real model answering
  a real prompt.
- LocalChatAndEmbeddingTest forces a genuine cold state with Ollama's
  keep_alive: 0 option (set via ChatModel.call(Prompt) -- ChatClient.options()
  does not carry a keepAlive override through to the request in this version)
  and confirms the unload actually happened via /api/ps before measuring a
  reload, rather than trusting whichever call happens to run first.
- On this quiet sandbox host, even a confirmed-cold reload of the 500MB model
  came back in single-digit milliseconds once the underlying image layers were
  cached -- eval (generation) time dominates total latency here, not loading.
  Captured, not asserted as universal: readers get scripts/bake-image.sh to
  get their own numbers.
- Embedding dimension (384, all-minilm) asserted deterministically.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FtpJvZfg4nvLvtzgJTDWpB
This commit is contained in:
Claude
2026-09-23 17:48:00 +00:00
parent 7fff4ddb99
commit faacda7079
14 changed files with 474 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ollama-local/pom.xml
+85
View File
@@ -0,0 +1,85 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.1</version>
<relativePath/>
</parent>
<groupId>com.ankurm</groupId>
<artifactId>ollama-local</artifactId>
<version>1.0.0</version>
<name>ollama-local</name>
<description>Running LLMs locally with Spring AI 2.0 and Ollama: chat and embeddings against a real local model, no API key, with a Testcontainers Ollama module baked for fast CI.</description>
<properties>
<java.version>25</java.version>
<spring-ai.version>2.0.1</spring-ai.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bom</artifactId>
<version>${spring-ai.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-ollama</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-testcontainers</artifactId>
<scope>test</scope>
</dependency>
<!-- Renamed from org.testcontainers:ollama in Testcontainers 1.x to org.testcontainers:testcontainers-ollama
in the 2.x line Spring Boot 4.1.1 manages (testcontainers-bom 2.0.5, imported transitively via
spring-boot-dependencies); the old artifactId still exists on Maven Central but is stuck at 1.21.4
and is NOT what this BOM resolves. -->
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-ollama</artifactId>
<scope>test</scope>
</dependency>
<!-- Also renamed in the 2.x line: org.testcontainers:junit-jupiter -> org.testcontainers:testcontainers-junit-jupiter. -->
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Duser.timezone=UTC -Dstdout.encoding=UTF-8 -Dfile.encoding=UTF-8</argLine>
</configuration>
</plugin>
</plugins>
</build>
</project>