asmhatre/spring-ai
1
0
Fork 0
Code Activity

Add docker-model-runner module: Spring AI OpenAI starter against Docker Model Runner and Ollama with a benchmark runner (time to first token, decode tokens/second, container memory, tool-call success); real docker/model-runner v1.2.9 CPU image and Ollama 0.40.13 on a shared 2-vCPU sandbox, no GPU

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017ic4b4VQVEJWetTT1dYzyJ
This commit is contained in:
asmhatre
2026-10-29 06:24:00 +00:00
co-authored by Claude Sonnet 5.5
parent 6d792f00e0
commit c97bcf9adf
18 changed files with 668 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker-model-runner/scripts/run-all.sh
Executable
+12
View File
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
# Regenerates output/dmr.txt and output/ollama.txt. Needs both servers up (scripts/start-dmr.sh,
# scripts/start-ollama.sh, or your own Docker Desktop Model Runner on :12434). Each engine is
# restarted first so its memory figure is a fresh container with only this one model loaded.
# Run it on an otherwise idle machine; the figures are only comparable within one run.
set -euo pipefail
cd "$(dirname "$0")/.."
ARGS="${BENCH_ARGS:---bench.repeats=3 --bench.max-tokens=48 --bench.tool-runs=10}"
docker restart ollama >/dev/null; sleep 5
scripts/bench.sh ollama http://localhost:11434/v1 qwen2.5:0.5b ollama $ARGS
docker restart dmr >/dev/null; sleep 5
scripts/bench.sh dmr http://localhost:12434/engines/v1 huggingface.co/qwen/qwen2.5-0.5b-instruct-gguf:Q4_K_M dmr $ARGS