Claude
faacda7079
- spring-ai-starter-model-ollama autoconfigures ChatModel/EmbeddingModel from spring.ai.ollama.* properties alone; no API key anywhere in this module. - org.testcontainers:ollama and org.testcontainers:junit-jupiter were both renamed in the Testcontainers 2.x line -- to org.testcontainers:testcontainers-ollama and org.testcontainers:testcontainers-junit-jupiter respectively -- confirmed by reading the real testcontainers-bom-2.0.5.pom that Spring Boot 4.1.1 imports (spring-boot-dependencies -> testcontainers.version=2.0.5). The pre-rename artifact IDs still exist on Maven Central but are stuck on the 1.x line. - Unlike every other module in this series, tests drive a real local model (qwen2.5:0.5b chat, all-minilm embeddings) via a Testcontainers-managed OllamaContainer started from a baked image (scripts/bake-image.sh), not a ScriptedChatModel -- the whole point of this post is a real model answering a real prompt. - LocalChatAndEmbeddingTest forces a genuine cold state with Ollama's keep_alive: 0 option (set via ChatModel.call(Prompt) -- ChatClient.options() does not carry a keepAlive override through to the request in this version) and confirms the unload actually happened via /api/ps before measuring a reload, rather than trusting whichever call happens to run first. - On this quiet sandbox host, even a confirmed-cold reload of the 500MB model came back in single-digit milliseconds once the underlying image layers were cached -- eval (generation) time dominates total latency here, not loading. Captured, not asserted as universal: readers get scripts/bake-image.sh to get their own numbers. - Embedding dimension (384, all-minilm) asserted deterministically. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01FtpJvZfg4nvLvtzgJTDWpB
47 lines
3.3 KiB
Markdown
47 lines
3.3 KiB
Markdown
# ollama-local
|
|
|
|
Companion code for [Run LLMs Locally with Spring AI and Ollama](https://ankurm.com/spring-ai-2-0-ollama-local/), part of the [Spring AI series](../README.md) on ankurm.com.
|
|
|
|
`spring-ai-starter-model-ollama` autoconfigures a `ChatModel` and an `EmbeddingModel` from `spring.ai.ollama.*` properties alone -- no API key anywhere in this module. Every test in [`LocalChatAndEmbeddingTest.java`](src/test/java/com/ankurm/ollamalocal/LocalChatAndEmbeddingTest.java) drives a real, local Ollama server started by Testcontainers, answering with a real small model (`qwen2.5:0.5b` for chat, `all-minilm` for embeddings) -- unlike every other module in this series, nothing here is scripted.
|
|
|
|
## Versions
|
|
|
|
| Component | Version |
|
|
|---|---|
|
|
| Spring Boot | 4.1.1 |
|
|
| Spring AI | 2.0.1 |
|
|
| Testcontainers | 2.0.5 |
|
|
| Java | 25 (LTS) |
|
|
|
|
`org.testcontainers:ollama` was renamed to **`org.testcontainers:testcontainers-ollama`** in the Testcontainers 2.x line, and `org.testcontainers:junit-jupiter` to **`org.testcontainers:testcontainers-junit-jupiter`** -- both confirmed by reading the real `testcontainers-bom-2.0.5.pom` that Spring Boot 4.1.1 imports. If you're copying an older Testcontainers-Ollama tutorial, the old artifact IDs still exist on Maven Central but are stuck on the 1.x line and are not what this BOM resolves.
|
|
|
|
## Quickstart
|
|
|
|
```bash
|
|
./scripts/bake-image.sh # once: pulls qwen2.5:0.5b + all-minilm into a local image
|
|
./scripts/run-all.sh # every time: runs the suite against that baked image
|
|
```
|
|
|
|
`bake-image.sh` is the real Testcontainers-recommended pattern for CI: pull the models into a container once, `docker commit` the result, and every subsequent test run starts a container that already has them on disk -- no registry pull, no network dependency, no per-run latency for the pull itself.
|
|
|
|
## What's here
|
|
|
|
| File | What it shows |
|
|
|---|---|
|
|
| [`OllamaLocalApplication.java`](src/main/java/com/ankurm/ollamalocal/OllamaLocalApplication.java) | The whole application: no manual `OllamaApi`/`OllamaChatModel` wiring, just the starter's autoconfiguration |
|
|
| [`config/ChatClientConfig.java`](src/main/java/com/ankurm/ollamalocal/config/ChatClientConfig.java) | Wraps the autoconfigured `ChatModel` in a `ChatClient`, same as every other module in this series |
|
|
| [`application.yml`](src/main/resources/application.yml) | The five `spring.ai.ollama.*` properties this module uses, with `pull-model-strategy: never` so the app fails fast instead of silently pulling gigabytes at startup |
|
|
| [`LocalChatAndEmbeddingTest.java`](src/test/java/com/ankurm/ollamalocal/LocalChatAndEmbeddingTest.java) | A Testcontainers-managed `OllamaContainer`, a confirmed model unload via `keep_alive: 0` + `/api/ps`, real chat calls, and a deterministic embedding-dimension assertion |
|
|
|
|
## Output files
|
|
|
|
| File | Captured from |
|
|
|---|---|
|
|
| `output/01-chat-reload-after-unload.txt` | `reloadAfterAConfirmedUnload` -- unloads the model, confirms via `/api/ps`, then reloads it |
|
|
| `output/02-chat-back-to-back-call.txt` | `backToBackCallReusesTheAlreadyLoadedModel` |
|
|
| `output/03-embedding-dimensions.txt` | `embeddingsAreDeterministicallySized` |
|
|
|
|
## Requirements
|
|
|
|
JDK 25, Maven, Docker. Run `scripts/bake-image.sh` before the test suite -- without a baked image present, `scripts/run-all.sh` refuses to start rather than silently pulling ~450MB of models over the network on every test run.
|