# docker-model-runner

Companion code for [Spring AI with Docker Model Runner vs Ollama](https://ankurm.com/spring-ai-docker-model-runner-vs-ollama/), part of the [Spring AI series](../README.md) on ankurm.com.

One Spring AI `OpenAiChatModel` (the `spring-ai-starter-model-openai` starter) pointed at two local servers by changing only `spring.ai.openai.base-url` and the model name. A benchmark runner sends the same prompts to each and reports time to first token, decode tokens/second, container memory (`docker stats`) and tool-call success over N runs.

Measured here: the real `docker/model-runner:latest` image (v1.2.9, llama.cpp CPU backend) and `ollama/ollama:latest` (0.40.13), both serving Qwen2.5-0.5B-Instruct at Q4_K_M, on a shared 2-vCPU sandbox with no GPU. Not measured: Docker Desktop's `docker model` CLI path and any GPU path.

| Component | Version |
|---|---|
| Spring Boot | 4.1.2 |
| Spring AI | 2.0.2 |
| Java | 25 (LTS) |

## Run

```bash
scripts/start-dmr.sh && scripts/start-ollama.sh   # EXTRA_CA=/path/ca.crt if HTTPS is intercepted
# (the model sizes in run-all.sh are the 0.5B ones; pull the same models in both engines)
scripts/run-all.sh                                # writes output/dmr.txt and output/ollama.txt
mvn test                                          # WireTest writes output/01-wire.txt
```

On Docker Desktop with `docker model` instead: `docker model pull hf.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M`, enable host-side TCP on port 12434, then `scripts/bench.sh dmr http://localhost:12434/engines/v1 <model id from /engines/v1/models> <container name>` (memory sampling needs a container name; skip it if there is none).

## Files

| File | What |
|---|---|
| `src/main/resources/application.yml` | the whole engine configuration |
| `Benchmark.java`, `WeatherTools.java` | throughput and tool-call harness |
| `WireTest.java` | records what the starter sends: path, key header, model field |
| `output/01-wire.txt` | request line, header and model name seen by a fake server |
| `output/02-engines.txt`, `03-dmr-extra-fields.txt` | `/models` listings, image ids, usage and timing fields |
| `output/dmr.txt`, `output/ollama.txt` | benchmark transcripts |