Add docker-model-runner module: Spring AI OpenAI starter against Docker Model Runner and Ollama with a benchmark runner (time to first token, decode tokens/second, container memory, tool-call success); real docker/model-runner v1.2.9 CPU image and Ollama 0.40.13 on a shared 2-vCPU sandbox, no GPU

Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_017ic4b4VQVEJWetTT1dYzyJ
2026-10-29 06:24:00 +00:00
parent 6d792f00e0
commit c97bcf9adf
18 changed files with 668 additions and 0 deletions
@@ -0,0 +1,34 @@
+#!/usr/bin/env bash
+# bench.sh <label> <base-url> <model> <container> [extra bench args]
+# Runs the Spring AI benchmark against one OpenAI-compatible server and samples the container's
+# memory with `docker stats` once a second while it runs. Writes output/<label>.txt.
+#   Docker Model Runner : bench.sh dmr    http://localhost:12434/engines/v1 hf.co/qwen/qwen2.5-0.5b-instruct-gguf:Q4_K_M dmr
+#   Ollama              : bench.sh ollama http://localhost:11434/v1          qwen2.5:0.5b ollama
+set -euo pipefail
+cd "$(dirname "$0")/.."
+LABEL=$1; URL=$2; MODEL=$3; CTR=$4; shift 4
+[ -f target/docker-model-runner-1.0.0.jar ] || mvn -B -q -T1 package -DskipTests
+mkdir -p output
+MEM=$(mktemp)
+( while true; do docker stats --no-stream --format '{{.MemUsage}}' "$CTR" | cut -d/ -f1 >> "$MEM"; sleep 1; done ) &
+SAMPLER=$!
+trap 'kill $SAMPLER 2>/dev/null || true' EXIT
+echo "idle container memory before: $(docker stats --no-stream --format '{{.MemUsage}}' "$CTR" | cut -d/ -f1)" > "output/$LABEL.mem.tmp"
+java -jar target/docker-model-runner-1.0.0.jar --bench.run=true --bench.label="$LABEL" \
+  --bench.out-file="output/$LABEL.txt" --spring.ai.openai.base-url="$URL" \
+  --spring.ai.openai.chat.options.model="$MODEL" "$@" > "output/$LABEL.log" 2>&1 || { tail -20 "output/$LABEL.log"; exit 1; }
+kill $SAMPLER 2>/dev/null || true
+python3 - "$MEM" "output/$LABEL.mem.tmp" "output/$LABEL.txt" <<'P'
+import sys,re
+def mb(s):
+    s=s.strip(); m=re.match(r'([\d.]+)(\w+)',s)
+    if not m: return None
+    v=float(m[1]); u=m[2]
+    return v*{'B':1/1048576,'KiB':1/1024,'MiB':1,'GiB':1024}[u]
+vals=[x for x in (mb(l) for l in open(sys.argv[1])) if x is not None]
+idle=open(sys.argv[2]).read().strip()
+with open(sys.argv[3],'a') as f:
+    f.write(f"\nmemory ({idle})\ncontainer memory during run: peak {max(vals):.0f} MiB, median {sorted(vals)[len(vals)//2]:.0f} MiB, {len(vals)} samples\n")
+P
+rm -f "output/$LABEL.mem.tmp" "output/$LABEL.log"
+cat "output/$LABEL.txt"