asmhatre/spring-ai
1
0
Fork 0
Code Activity

Add docker-model-runner module: Spring AI OpenAI starter against Docker Model Runner and Ollama with a benchmark runner (time to first token, decode tokens/second, container memory, tool-call success); real docker/model-runner v1.2.9 CPU image and Ollama 0.40.13 on a shared 2-vCPU sandbox, no GPU

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017ic4b4VQVEJWetTT1dYzyJ
This commit is contained in:
asmhatre
2026-10-29 06:24:00 +00:00
co-authored by Claude Sonnet 5.5
parent 6d792f00e0
commit c97bcf9adf
18 changed files with 668 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker-model-runner/output/01-wire.txt
+8
View File
@@ -0,0 +1,8 @@
base-url = http://127.0.0.1:<port>/engines/v1
spring.ai.openai.chat.options.model = configured-model
call 1, model.call(new Prompt("ping"))
POST /engines/v1/chat/completions | Authorization: Bearer not-needed | model: configured-model
call 2, same prompt with OpenAiChatOptions.builder().temperature(0.0).build()
POST /engines/v1/chat/completions | Authorization: Bearer not-needed | model: gpt-5-mini
docker-model-runner/output/02-engines.txt
+12
View File
@@ -0,0 +1,12 @@
# GET http://localhost:12434/engines/v1/models (Docker Model Runner, docker/model-runner v1.2.9 image)
{"object":"list","data":[{"id":"huggingface.co/qwen/qwen2.5-0.5b-instruct-gguf:Q4_K_M","object":"model","created":1726813224,"owned_by":"docker","dmr":{"context_window":32768,"architecture":"qwen2","parameters":"630.17M","quantization":"MOSTLY_Q4_K_M","size":"462.96MiB"}}]}
# GET http://localhost:11434/v1/models (Ollama)
{"object":"list","data":[{"id":"qwen2.5:0.5b","object":"model","created":1793253801,"owned_by":"library"},{"id":"qwen2.5:1.5b","object":"model","created":1793253450,"owned_by":"library"}]}
# docker image digests
docker/model-runner:latest sha256:60e6a14d8cc331fab3e51d7b78bba98274b4d28573f4e0b345e475d52d167869
ollama/ollama:latest sha256:b2f0122c476bc9fe7ce679d8bb4bd555351e25e2554eefb156f61cb0a89cf429
ollama version is 0.40.13
docker-model-runner/output/03-dmr-extra-fields.txt
+4
View File
@@ -0,0 +1,4 @@
usage {'completion_tokens': 16, 'prompt_tokens': 31, 'total_tokens': 47, 'prompt_tokens_details': {'cached_tokens': 24}}
timings keys ['cache_n', 'predicted_ms', 'predicted_n', 'predicted_per_second', 'predicted_per_token_ms', 'prompt_ms', 'prompt_n', 'prompt_per_second', 'prompt_per_token_ms']
ollama usage {'prompt_tokens': 31, 'prompt_tokens_details': {'cached_tokens': 24}, 'completion_tokens': 10, 'total_tokens': 41}
ollama top-level keys ['choices', 'created', 'id', 'model', 'object', 'system_fingerprint', 'usage']
docker-model-runner/output/dmr.txt
+56
View File
@@ -0,0 +1,56 @@
engine label: dmr
java: 25.0.5+8-LTS
first request (includes model load): ttft 5018 ms, 48 completion tokens
throughput, 5 prompts x 3 repeats, temperature 0, max_tokens 48
p rep ttft_ms decode_tps wall_tps prompt compl
1 1 514 19.97 16.70 41 48
1 2 74 18.03 17.88 41 48
1 3 91 17.50 17.26 41 48
2 1 350 16.35 14.85 45 48
2 2 488 13.79 12.29 45 48
2 3 82 14.19 14.12 45 48
3 1 315 17.07 15.60 43 48
3 2 75 18.15 17.99 43 48
3 3 91 17.61 17.36 43 48
4 1 337 16.42 14.97 46 48
4 2 87 17.94 17.70 46 48
4 3 101 18.15 17.80 46 48
5 1 308 17.45 15.96 45 48
5 2 76 18.30 18.09 45 48
5 3 89 18.15 17.88 45 48
median ttft_ms 91 | median decode_tps 17.61 | median wall_tps 17.26 | n=15
tool calling (one tool), 10 runs, temperature 0.7 so runs differ
question: What is the temperature in Oslo in Celsius? Use the tools.
expected: currentTemperature(Oslo), answer contains 17
run 1 wrong or missing call calls=[celsiusToFahrenheit(12)] answer=The temperature in Oslo is 54 degrees Fahrenheit.
run 2 ok calls=[currentTemperature(Oslo)] answer=The temperature in Oslo is 17 degrees Celsius.
run 3 wrong or missing call calls=[celsiusToFahrenheit(12)] answer=The temperature in Oslo is 54 degrees Celsius.
run 4 wrong or missing call calls=[celsiusToFahrenheit(12)] answer=The temperature in Oslo is 54 degrees Fahrenheit.
run 5 wrong or missing call calls=[celsiusToFahrenheit(12)] answer=The temperature in Oslo is 54 degrees Fahrenheit.
run 6 wrong or missing call calls=[celsiusToFahrenheit(0)] answer=The temperature in Oslo is 32 degrees Celsius.
run 7 wrong or missing call calls=[celsiusToFahrenheit(10)] answer=The temperature in Oslo is 50 degrees Fahrenheit.
run 8 wrong or missing call calls=[celsiusToFahrenheit(0)] answer=The temperature in Oslo is 32 degrees Fahrenheit.
run 9 wrong or missing call calls=[celsiusToFahrenheit(14)] answer=The temperature in Oslo is 57 degrees Fahrenheit.
run 10 wrong or missing call calls=[celsiusToFahrenheit(12)] answer=The temperature in Oslo is 54 degrees Fahrenheit.
tool-call success (one tool): 1/10 (no call 0, wrong/missing call 9, wrong answer 0, errors 0)
tool calling (two tools in sequence), 10 runs, temperature 0.7 so runs differ
question: What is the temperature in Pune in Fahrenheit? Use the tools.
expected: currentTemperature(Pune), celsiusToFahrenheit(31), answer contains 88
run 1 wrong or missing call calls=[currentTemperature(Pune)] answer=The temperature in Pune is 31 degrees Fahrenheit.
run 2 wrong or missing call calls=[celsiusToFahrenheit(22)] answer=The temperature in Pune is 72 degrees Fahrenheit.
run 3 wrong or missing call calls=[currentTemperature(Pune)] answer=The temperature in Pune is 31 degrees Fahrenheit.
run 4 wrong or missing call calls=[celsiusToFahrenheit(25)] answer=The temperature in Pune is 77 degrees Fahrenheit.
run 5 wrong or missing call calls=[celsiusToFahrenheit(15)] answer=The temperature in Pune is 59 degrees Fahrenheit.
run 6 wrong or missing call calls=[celsiusToFahrenheit(0)] answer=The temperature in Pune is 32 degrees Fahrenheit.
run 7 wrong or missing call calls=[currentTemperature(Pune)] answer=The temperature in Pune is 31 degrees Fahrenheit.
run 8 wrong or missing call calls=[celsiusToFahrenheit(15)] answer=The temperature in Pune is 59 degrees Fahrenheit.
run 9 wrong or missing call calls=[celsiusToFahrenheit(32)] answer=The temperature in Pune is 90 degrees Fahrenheit.
run 10 wrong or missing call calls=[celsiusToFahrenheit(15)] answer=The temperature in Pune is 59 degrees Fahrenheit.
tool-call success (two tools in sequence): 0/10 (no call 0, wrong/missing call 10, wrong answer 0, errors 0)
memory (idle container memory before: 3.695MiB)
container memory during run: peak 546 MiB, median 511 MiB, 58 samples
docker-model-runner/output/ollama.txt
+56
View File
@@ -0,0 +1,56 @@
engine label: ollama
java: 25.0.5+8-LTS
first request (includes model load): ttft 4946 ms, 48 completion tokens
throughput, 5 prompts x 3 repeats, temperature 0, max_tokens 48
p rep ttft_ms decode_tps wall_tps prompt compl
1 1 247 17.59 16.40 41 48
1 2 66 18.43 18.32 41 48
1 3 63 18.02 17.93 41 48
2 1 289 18.99 17.35 45 48
2 2 92 18.93 18.58 45 48
2 3 65 18.56 18.43 45 48
3 1 322 18.36 16.63 43 48
3 2 58 20.94 20.79 43 48
3 3 84 17.65 17.46 43 48
4 1 287 19.89 18.07 46 48
4 2 68 19.30 19.15 46 48
4 3 70 18.57 18.44 46 48
5 1 282 17.87 16.46 45 48
5 2 90 18.76 18.44 45 48
5 3 68 18.74 18.55 45 48
median ttft_ms 84 | median decode_tps 18.57 | median wall_tps 18.32 | n=15
tool calling (one tool), 10 runs, temperature 0.7 so runs differ
question: What is the temperature in Oslo in Celsius? Use the tools.
expected: currentTemperature(Oslo), answer contains 17
run 1 ok calls=[currentTemperature(Oslo)] answer=The current temperature in Oslo is 17 degrees Celsius.
run 2 wrong or missing call calls=[celsiusToFahrenheit(0)] answer=The temperature in Oslo, Norway, is 32 degrees Celsius.
run 3 wrong or missing call calls=[celsiusToFahrenheit(20)] answer=The temperature in Oslo is 68 degrees Fahrenheit.
run 4 wrong or missing call calls=[celsiusToFahrenheit(0)] answer=The temperature in Oslo is 32 degrees Celsius.
run 5 wrong or missing call calls=[celsiusToFahrenheit(10)] answer=The temperature in Oslo is currently 50 degrees Fahrenheit.
run 6 wrong or missing call calls=[celsiusToFahrenheit(20)] answer=The temperature in Oslo, at 20 degrees Celsius, is 68 degree
run 7 wrong or missing call calls=[celsiusToFahrenheit(0)] answer=The temperature in Oslo in Celsius is 32 degrees.
run 8 wrong or missing call calls=[celsiusToFahrenheit(4)] answer=The temperature in Oslo (Oslo) is currently 39 degrees Celsi
run 9 wrong or missing call calls=[celsiusToFahrenheit(6)] answer=The temperature in Oslo is 43 degrees Fahrenheit.
run 10 wrong or missing call calls=[celsiusToFahrenheit(0)] answer=The temperature in Oslo is 32 degrees Celsius.
tool-call success (one tool): 1/10 (no call 0, wrong/missing call 9, wrong answer 0, errors 0)
tool calling (two tools in sequence), 10 runs, temperature 0.7 so runs differ
question: What is the temperature in Pune in Fahrenheit? Use the tools.
expected: currentTemperature(Pune), celsiusToFahrenheit(31), answer contains 88
run 1 wrong or missing call calls=[celsiusToFahrenheit(22)] answer=The temperature in Pune is 72 degrees Fahrenheit.
run 2 wrong or missing call calls=[celsiusToFahrenheit(32)] answer=The temperature in Pune is currently 90 degrees Fahrenheit.
run 3 wrong or missing call calls=[celsiusToFahrenheit(26)] answer=The temperature in Pune is 79 degrees Fahrenheit.
run 4 wrong or missing call calls=[celsiusToFahrenheit(25)] answer=The temperature in Pune is 77 degrees Fahrenheit.
run 5 wrong or missing call calls=[celsiusToFahrenheit(26)] answer=The temperature in Pune (Pune) is 79 degrees Fahrenheit. It'
run 6 wrong or missing call calls=[celsiusToFahrenheit(23)] answer=The temperature in Pune is 73 degrees Fahrenheit.
run 7 wrong or missing call calls=[celsiusToFahrenheit(23)] answer=The temperature in Pune is 73 degrees Fahrenheit.
run 8 wrong or missing call calls=[celsiusToFahrenheit(15)] answer=The temperature in Pune is 59 degrees Fahrenheit.
run 9 wrong or missing call calls=[celsiusToFahrenheit(0)] answer=The temperature in Pune is 32 degrees Fahrenheit.
run 10 wrong or missing call calls=[celsiusToFahrenheit(22)] answer=The temperature in Pune is 72 degrees Fahrenheit.
tool-call success (two tools in sequence): 0/10 (no call 0, wrong/missing call 10, wrong answer 0, errors 0)
memory (idle container memory before: 12.15MiB)
container memory during run: peak 510 MiB, median 509 MiB, 55 samples