Every other post in this series calls out to a hosted model. That means an API key in an environment variable, a network round trip on every request, and a per-token bill running in the background even while you’re just trying things out. This article is the exception: spring-ai-starter-model-ollama talks to Ollama, a server that runs models on your own machine, over plain HTTP, with nothing to sign up for. No api-key property appears anywhere in this article’s companion module — not because it’s hidden, but because there’s genuinely nothing to put there.
Running locally trades one set of problems for another. A model small enough to run well on a laptop’s CPU is not a model you’d put in front of a paying customer for anything demanding, and getting it into a CI pipeline without your test suite silently downloading gigabytes on every run takes a specific pattern most tutorials skip. This article covers both: a real local ChatClient and EmbeddingModel, and a Testcontainers setup that never touches the network once you’ve built it.
Versions. Spring Boot 4.1.1 and Spring AI 2.0.1, on Java 25 (LTS) — the same baseline as the rest of this series. Testcontainers 2.0.5, which is what Spring Boot 4.1.1 itself manages via its inherited testcontainers-bom import. Two Testcontainers artifact IDs changed between the 1.x and 2.x lines in ways that will silently break an older tutorial’s pom.xml — covered below, confirmed by reading the real BOM file rather than assuming the old names still apply.
A ChatClient with nothing to configure but a URL
Every other module in this series’ application.yml has an api-key line. This one doesn’t:
Source: application.yml. spring-ai-starter-model-ollama reads these five properties and autoconfigures both an OllamaChatModel and an OllamaEmbeddingModel — there’s no manual OllamaApi or @Bean wiring anywhere in this module’s main source, same as the rest of the series wraps whatever ChatModel autoconfiguration produced in a ChatClient:
Source: ChatClientConfig.java.
qwen2.5:0.5b is a 0.5-billion-parameter chat model, quantized to roughly 400 MB on disk — small enough to run on a CPU-only machine without waiting minutes per reply. all-minilm is a 45 MB embedding model producing 384-dimension vectors. Neither is what you’d deploy for a demanding production workload; both are genuinely useful for local development, offline demos, and exactly the kind of CI test this article builds toward.
init.pull-model-strategy: never is actually Ollama autoconfiguration’s own default, written out explicitly here rather than relied on silently — the alternative is when_missing or always, both of which will pull a model at application-startup time if it isn’t already present locally. never means a missing model fails the very first request instead of silently downloading several hundred megabytes the first time someone runs the app.
Testing against a real model without a network dependency
A test suite that calls out to a real model over the network is a test suite that’s occasionally slow, occasionally flaky, and — the first time it runs in a fresh CI container — pulls hundreds of megabytes before the first assertion even runs. The fix is the same one Testcontainers recommends for any slow-to-provision dependency: build the state you need into an image once, and let every test run start a container that already has it.
#!/usr/bin/env bash
docker run -d --name "$CONTAINER_NAME" ollama/ollama:latest
until docker exec "$CONTAINER_NAME" ollama list >/dev/null 2>&1; do sleep 1; done
docker exec "$CONTAINER_NAME" ollama pull qwen2.5:0.5b
docker exec "$CONTAINER_NAME" ollama pull all-minilm
docker commit "$CONTAINER_NAME" "$IMAGE_TAG"
Source: bake-image.sh (trimmed — the real file adds a container-name-per-run and a cleanup trap). Run it once, and ollama-baked-qwen05b-minilm:local has both models already on disk. Every test run after that starts a container from that image with no registry pull at all.
@Container
static final OllamaContainer OLLAMA = new OllamaContainer(
DockerImageName.parse("ollama-baked-qwen05b-minilm:local")
.asCompatibleSubstituteFor("ollama/ollama"));
@DynamicPropertySource
static void ollamaProperties(DynamicPropertyRegistry registry) {
registry.add("spring.ai.ollama.base-url", OLLAMA::getEndpoint);
}
Source: LocalChatAndEmbeddingTest.java. asCompatibleSubstituteFor("ollama/ollama") is what lets OllamaContainer — which otherwise expects an image tagged as the official one — accept a locally-built, differently-named image instead.
Two artifact IDs changed in Testcontainers 2.x.org.testcontainers:ollama became org.testcontainers:testcontainers-ollama, and org.testcontainers:junit-jupiter became org.testcontainers:testcontainers-junit-jupiter. Both old IDs still exist on Maven Central, still resolve, and are stuck permanently on the 1.x line — mvn dependency:resolve won’t warn you, it’ll just quietly give you the wrong major version. This was confirmed by downloading and reading the real testcontainers-bom-2.0.5.pom that Spring Boot 4.1.1 imports (via its own spring-boot-dependencies), not by trusting an older tutorial’s pom.xml.
Going deeper on this section
Companion repo: pom.xml (see the comments next to both renamed dependencies)
Confirming a model actually unloaded, not just assuming it
Ollama keeps a model resident in memory for a while after each response, by default 5 minutes, so a second call arriving soon after the first reuses whatever’s already loaded instead of reading weights off disk again. The keep_alive request option controls this directly, and setting it to "0" tells Ollama to unload the model the moment the current call finishes — the same knob you’d reach for in production to free RAM between bursts of unrelated traffic.
Source: LocalChatAndEmbeddingTest.java. keepAlive passes through ChatClient‘s per-call options() exactly like any other OllamaChatOptions field — no special-casing needed.
Rather than trust that the unload happened, the test asks Ollama’s own registry endpoint:
/api/ps immediately after the unload call: {"models":[]}
prompt: "Reply with a single short sentence: why do developers like small local models?"
response: Developers often appreciate small local models because they are more efficient and can provide quicker, more relevant results for their applications.
total-duration: 1076ms
load-duration: 2ms
prompt-eval-count: 44, prompt-eval-duration: 41ms
eval-count: 27, eval-duration: 1028ms
26.25 tokens/sec (eval-count / eval-duration)
Output: 01-chat-reload-after-unload.txt. {"models":[]} is Ollama’s own confirmation the registry was genuinely empty — this is a real reload, not a call that got lucky and hit an already-warm model. And on this sandbox’s quiet, otherwise-idle hardware, that reload’s load-duration came back at a couple of milliseconds. A back-to-back call straight after shows the same thing:
prompt: "Reply with a single short sentence: what is Testcontainers for?"
response: Testcontainers is a containerization platform that simplifies the process of building and deploying applications quickly and efficiently.
total-duration: 662ms
load-duration: 2ms
prompt-eval-count: 42, prompt-eval-duration: 47ms
eval-count: 16, eval-duration: 607ms
26.33 tokens/sec (eval-count / eval-duration)
Output: 02-chat-back-to-back-call.txt. Loading a ~400 MB q4-quantized model off an already-cached disk image turned out to be close to free here — both runs above spend almost all of their total-duration in eval-duration, the actual token generation, at a consistent ~26 tokens/sec on this sandbox’s 2-vCPU, GPU-less hardware. That’s a real number from this environment, not a universal one: a machine whose disk cache is genuinely cold, or a larger model, will spend a much bigger share of its time in load-duration instead. scripts/bake-image.sh plus this test file is the reproducible way to get your own numbers rather than trusting either this article’s or anyone else’s.
Going deeper: the real ChatResponseMetadata keys behind these numbers
Every number above comes from ChatResponse.getMetadata(), an ordinary ResponseMetadata map keyed by string. Confirmed directly from OllamaChatModel‘s constant pool via javap -v, the real keys are total-duration, load-duration, prompt-eval-count, prompt-eval-duration, eval-count, and eval-duration — the four duration keys come back as java.time.Duration, and the two count keys as Integer. There’s no separate DTO to look up; metadata.<Duration>get("load-duration") is the whole API.
spring.ai.ollama.embedding.options.model autoconfigures a second model, entirely independent of the chat one, behind the same EmbeddingModel interface every other embedding provider in Spring AI implements:
float[] vector = embeddingModel.embed("Spring AI can run entirely against a local Ollama server.");
// vector.length == 384
// embeddingModel.dimensions() == 384
Output: 03-embedding-dimensions.txt. Unlike the chat responses earlier in this article, this output is fully deterministic and safe to assert on exactly — the same input text always produces the same 384 floating-point values from the same model, which is exactly the property a vector index (this series’ RAG article uses pgvector) depends on.
pull-model-strategy: never means a request against a model that was never pulled fails outright — deliberately, rather than silently pulling several hundred megabytes the first time a teammate runs the app or a CI job spins up a fresh container. That’s a real, reachable failure mode worth knowing the shape of before it shows up unannounced.
Property
Default
What it controls
spring.ai.ollama.init.pull-model-strategy
never
never fails fast on a missing model; when_missing pulls it once at startup if absent; always re-pulls (and re-verifies) on every startup
spring.ai.ollama.init.max-retries
0
retry attempts for a failed pull, when a strategy other than never is in effect
spring.ai.ollama.init.timeout
5m
how long to wait for a pull to finish before giving up
spring.ai.ollama.init.chat.additional-models
—
extra models to initialize beyond the one chat.options.model names (an init.embedding.additional-models mirror exists too)
Confirmed directly from this starter’s spring-configuration-metadata.json, not the reference docs’ prose — the same file an IDE reads to autocomplete application.yml. never being the strategy’s own default, not something this module overrides, is worth calling out on its own: Spring AI’s Ollama starter ships fail-fast by default, and reaching for when_missing is an opt-in decision, not something that happens to a project by accident.
Pulling at startup and pulling in a test suite are different problems.when_missing is a reasonable choice for a developer’s own machine on first run. It’s a much worse one for CI, where “missing” is true on every fresh runner and a multi-hundred-megabyte pull becomes part of your build’s critical path, every single time. The baked-image pattern earlier in this article exists specifically so a test suite never has to make that choice at all.
Should you run a model locally at all?
For a lot of what teams actually spend time on, yes: local development against a real model instead of a mock, a demo that works on a flight with no wifi, a CI suite that exercises real prompt-and-parse logic without a per-run API bill, and prototyping a prompt before deciding it’s worth the latency and cost of a much larger hosted model. None of that requires frontier-model quality.
For anything where response quality is the product — a customer-facing assistant, complex multi-step reasoning, anything competing with what a much larger hosted model can do — a 0.5B local model is very much the wrong tool, and no amount of prompt engineering changes that. The honest reason to reach for Ollama in production at all is when the workload itself is the kind small, specialized models are good at (classification, extraction against a narrow schema, an offline-first application with no reliable network), not as a drop-in cost-saving swap for a frontier model.
The same ChatClient code either way. Nothing in this series’ ChatClientConfig pattern is Ollama-specific — swap the starter dependency and the spring.ai.* properties, and the same ChatClient-based application code from this series’ very first article runs against a hosted model instead. Local-first development and a hosted-model production deployment aren’t a rewrite away from each other; they’re a properties file away.
No Comments yet!