Files

T

Claude faacda7079 Add ollama-local module: chat and embeddings against a real local Ollama server

- spring-ai-starter-model-ollama autoconfigures ChatModel/EmbeddingModel from
  spring.ai.ollama.* properties alone; no API key anywhere in this module.
- org.testcontainers:ollama and org.testcontainers:junit-jupiter were both
  renamed in the Testcontainers 2.x line -- to org.testcontainers:testcontainers-ollama
  and org.testcontainers:testcontainers-junit-jupiter respectively -- confirmed by
  reading the real testcontainers-bom-2.0.5.pom that Spring Boot 4.1.1 imports
  (spring-boot-dependencies -> testcontainers.version=2.0.5). The pre-rename
  artifact IDs still exist on Maven Central but are stuck on the 1.x line.
- Unlike every other module in this series, tests drive a real local model
  (qwen2.5:0.5b chat, all-minilm embeddings) via a Testcontainers-managed
  OllamaContainer started from a baked image (scripts/bake-image.sh), not a
  ScriptedChatModel -- the whole point of this post is a real model answering
  a real prompt.
- LocalChatAndEmbeddingTest forces a genuine cold state with Ollama's
  keep_alive: 0 option (set via ChatModel.call(Prompt) -- ChatClient.options()
  does not carry a keepAlive override through to the request in this version)
  and confirms the unload actually happened via /api/ps before measuring a
  reload, rather than trusting whichever call happens to run first.
- On this quiet sandbox host, even a confirmed-cold reload of the 500MB model
  came back in single-digit milliseconds once the underlying image layers were
  cached -- eval (generation) time dominates total latency here, not loading.
  Captured, not asserted as universal: readers get scripts/bake-image.sh to
  get their own numbers.
- Embedding dimension (384, all-minilm) asserted deterministically.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01FtpJvZfg4nvLvtzgJTDWpB

2026-09-23 17:48:00 +00:00

3.3 KiB

Raw Permalink Blame History

ollama-local

Companion code for Run LLMs Locally with Spring AI and Ollama, part of the Spring AI series on ankurm.com.

spring-ai-starter-model-ollama autoconfigures a ChatModel and an EmbeddingModel from spring.ai.ollama.* properties alone -- no API key anywhere in this module. Every test in LocalChatAndEmbeddingTest.java drives a real, local Ollama server started by Testcontainers, answering with a real small model (qwen2.5:0.5b for chat, all-minilm for embeddings) -- unlike every other module in this series, nothing here is scripted.

Versions

Component	Version
Spring Boot	4.1.1
Spring AI	2.0.1
Testcontainers	2.0.5
Java	25 (LTS)

org.testcontainers:ollama was renamed to org.testcontainers:testcontainers-ollama in the Testcontainers 2.x line, and org.testcontainers:junit-jupiter to org.testcontainers:testcontainers-junit-jupiter -- both confirmed by reading the real testcontainers-bom-2.0.5.pom that Spring Boot 4.1.1 imports. If you're copying an older Testcontainers-Ollama tutorial, the old artifact IDs still exist on Maven Central but are stuck on the 1.x line and are not what this BOM resolves.

Quickstart

./scripts/bake-image.sh   # once: pulls qwen2.5:0.5b + all-minilm into a local image
./scripts/run-all.sh      # every time: runs the suite against that baked image

bake-image.sh is the real Testcontainers-recommended pattern for CI: pull the models into a container once, docker commit the result, and every subsequent test run starts a container that already has them on disk -- no registry pull, no network dependency, no per-run latency for the pull itself.

What's here

File	What it shows
`OllamaLocalApplication.java`	The whole application: no manual `OllamaApi`/`OllamaChatModel` wiring, just the starter's autoconfiguration
`config/ChatClientConfig.java`	Wraps the autoconfigured `ChatModel` in a `ChatClient`, same as every other module in this series
`application.yml`	The five `spring.ai.ollama.*` properties this module uses, with `pull-model-strategy: never` so the app fails fast instead of silently pulling gigabytes at startup
`LocalChatAndEmbeddingTest.java`	A Testcontainers-managed `OllamaContainer`, a confirmed model unload via `keep_alive: 0` + `/api/ps`, real chat calls, and a deterministic embedding-dimension assertion

Output files

File	Captured from
`output/01-chat-reload-after-unload.txt`	`reloadAfterAConfirmedUnload` -- unloads the model, confirms via `/api/ps`, then reloads it
`output/02-chat-back-to-back-call.txt`	`backToBackCallReusesTheAlreadyLoadedModel`
`output/03-embedding-dimensions.txt`	`embeddingsAreDeterministicallySized`

Requirements

JDK 25, Maven, Docker. Run scripts/bake-image.sh before the test suite -- without a baked image present, scripts/run-all.sh refuses to start rather than silently pulling ~450MB of models over the network on every test run.

3.3 KiB Raw Permalink Blame History