Claude
faacda7079
- spring-ai-starter-model-ollama autoconfigures ChatModel/EmbeddingModel from spring.ai.ollama.* properties alone; no API key anywhere in this module. - org.testcontainers:ollama and org.testcontainers:junit-jupiter were both renamed in the Testcontainers 2.x line -- to org.testcontainers:testcontainers-ollama and org.testcontainers:testcontainers-junit-jupiter respectively -- confirmed by reading the real testcontainers-bom-2.0.5.pom that Spring Boot 4.1.1 imports (spring-boot-dependencies -> testcontainers.version=2.0.5). The pre-rename artifact IDs still exist on Maven Central but are stuck on the 1.x line. - Unlike every other module in this series, tests drive a real local model (qwen2.5:0.5b chat, all-minilm embeddings) via a Testcontainers-managed OllamaContainer started from a baked image (scripts/bake-image.sh), not a ScriptedChatModel -- the whole point of this post is a real model answering a real prompt. - LocalChatAndEmbeddingTest forces a genuine cold state with Ollama's keep_alive: 0 option (set via ChatModel.call(Prompt) -- ChatClient.options() does not carry a keepAlive override through to the request in this version) and confirms the unload actually happened via /api/ps before measuring a reload, rather than trusting whichever call happens to run first. - On this quiet sandbox host, even a confirmed-cold reload of the 500MB model came back in single-digit milliseconds once the underlying image layers were cached -- eval (generation) time dominates total latency here, not loading. Captured, not asserted as universal: readers get scripts/bake-image.sh to get their own numbers. - Embedding dimension (384, all-minilm) asserted deterministically. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01FtpJvZfg4nvLvtzgJTDWpB
36 lines
1.4 KiB
Bash
Executable File
36 lines
1.4 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Pulls ollama/ollama:latest, starts it once, pulls qwen2.5:0.5b (chat, ~400MB) and all-minilm
|
|
# (embeddings, ~45MB) inside the running container, then `docker commit`s the result to a local
|
|
# image tag. This is the real Testcontainers-recommended pattern for CI: bake the models into
|
|
# an image once, and every subsequent test run starts a container that already has them on
|
|
# disk -- no registry pull, no flaky network dependency, no per-run latency for the pull itself.
|
|
#
|
|
# Run this once before scripts/run-all.sh, or whenever you want to refresh the baked models.
|
|
set -euo pipefail
|
|
|
|
IMAGE_TAG="${1:-ollama-baked-qwen05b-minilm:local}"
|
|
CONTAINER_NAME="ollama-bake-$$"
|
|
|
|
echo "Pulling ollama/ollama:latest..."
|
|
docker pull ollama/ollama:latest
|
|
|
|
echo "Starting a container to pull models into..."
|
|
docker run -d --name "$CONTAINER_NAME" ollama/ollama:latest
|
|
trap 'docker rm -f "$CONTAINER_NAME" >/dev/null 2>&1 || true' EXIT
|
|
|
|
echo "Waiting for the Ollama server to accept connections..."
|
|
until docker exec "$CONTAINER_NAME" ollama list >/dev/null 2>&1; do
|
|
sleep 1
|
|
done
|
|
|
|
echo "Pulling qwen2.5:0.5b (chat)..."
|
|
docker exec "$CONTAINER_NAME" ollama pull qwen2.5:0.5b
|
|
|
|
echo "Pulling all-minilm (embeddings)..."
|
|
docker exec "$CONTAINER_NAME" ollama pull all-minilm
|
|
|
|
echo "Committing to ${IMAGE_TAG}..."
|
|
docker commit "$CONTAINER_NAME" "$IMAGE_TAG"
|
|
|
|
echo "Done. ${IMAGE_TAG} now has both models baked in -- scripts/run-all.sh will not touch the network."
|