spring-ai/docker-model-runner/README.md

docker-model-runner

Companion code for Spring AI with Docker Model Runner vs Ollama, part of the Spring AI series on ankurm.com.

One Spring AI OpenAiChatModel (the spring-ai-starter-model-openai starter) pointed at two local servers by changing only spring.ai.openai.base-url and the model name. A benchmark runner sends the same prompts to each and reports time to first token, decode tokens/second, container memory (docker stats) and tool-call success over N runs.

Measured here: the real docker/model-runner:latest image (v1.2.9, llama.cpp CPU backend) and ollama/ollama:latest (0.40.13), both serving Qwen2.5-0.5B-Instruct at Q4_K_M, on a shared 2-vCPU sandbox with no GPU. Not measured: Docker Desktop's docker model CLI path and any GPU path.

Component	Version
Spring Boot	4.1.2
Spring AI	2.0.2
Java	25 (LTS)

Run

scripts/start-dmr.sh && scripts/start-ollama.sh   # EXTRA_CA=/path/ca.crt if HTTPS is intercepted
# (the model sizes in run-all.sh are the 0.5B ones; pull the same models in both engines)
scripts/run-all.sh                                # writes output/dmr.txt and output/ollama.txt
mvn test                                          # WireTest writes output/01-wire.txt

On Docker Desktop with docker model instead: docker model pull hf.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M, enable host-side TCP on port 12434, then scripts/bench.sh dmr http://localhost:12434/engines/v1 <model id from /engines/v1/models> <container name> (memory sampling needs a container name; skip it if there is none).

Files

File	What
src/main/resources/application.yml	the whole engine configuration
Benchmark.java, WeatherTools.java	throughput and tool-call harness
WireTest.java	records what the starter sends: path, key header, model field
output/01-wire.txt	request line, header and model name seen by a fake server
output/02-engines.txt, 03-dmr-extra-fields.txt	/models listings, image ids, usage and timing fields
output/dmr.txt, output/ollama.txt	benchmark transcripts