Add ollama-local module: chat and embeddings against a real local Ollama server
- spring-ai-starter-model-ollama autoconfigures ChatModel/EmbeddingModel from spring.ai.ollama.* properties alone; no API key anywhere in this module. - org.testcontainers:ollama and org.testcontainers:junit-jupiter were both renamed in the Testcontainers 2.x line -- to org.testcontainers:testcontainers-ollama and org.testcontainers:testcontainers-junit-jupiter respectively -- confirmed by reading the real testcontainers-bom-2.0.5.pom that Spring Boot 4.1.1 imports (spring-boot-dependencies -> testcontainers.version=2.0.5). The pre-rename artifact IDs still exist on Maven Central but are stuck on the 1.x line. - Unlike every other module in this series, tests drive a real local model (qwen2.5:0.5b chat, all-minilm embeddings) via a Testcontainers-managed OllamaContainer started from a baked image (scripts/bake-image.sh), not a ScriptedChatModel -- the whole point of this post is a real model answering a real prompt. - LocalChatAndEmbeddingTest forces a genuine cold state with Ollama's keep_alive: 0 option (set via ChatModel.call(Prompt) -- ChatClient.options() does not carry a keepAlive override through to the request in this version) and confirms the unload actually happened via /api/ps before measuring a reload, rather than trusting whichever call happens to run first. - On this quiet sandbox host, even a confirmed-cold reload of the 500MB model came back in single-digit milliseconds once the underlying image layers were cached -- eval (generation) time dominates total latency here, not loading. Captured, not asserted as universal: readers get scripts/bake-image.sh to get their own numbers. - Embedding dimension (384, all-minilm) asserted deterministically. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01FtpJvZfg4nvLvtzgJTDWpB
This commit is contained in:
@@ -11,5 +11,6 @@ Runnable companion code for the Spring AI articles on [ankurm.com](https://ankur
|
||||
| [`mcp-secure/`](mcp-secure) | The mcp-server article's order-lookup tools behind a real OAuth2 resource server: JWT validation, one scope per tool via `@PreAuthorize`, unauthenticated tool discovery rejected outright, and every call audit-logged through MDC -- denials included. Spring Boot 4.1.1, Spring AI 2.0.1, Spring Security 7.1.1, Java 25. | [Securing an MCP Server with Spring Security 7](https://ankurm.com/spring-ai-2-0-mcp-server-security/) |
|
||||
| [`tool-calling/`](tool-calling) | `@Tool` methods, `ToolCallingAdvisor` (the advisor-layer replacement for Spring AI 1.x's per-model tool loop), `returnDirect`, `ToolContext`, and `ToolSearchToolCallingAdvisor` for progressive disclosure across a 230-tool synthetic library -- every test driven by a hand-written `ScriptedChatModel`, no live model anywhere. Spring Boot 4.1.1, Spring AI 2.0.1, Java 25. | [Tool Calling in Spring AI 2.0](https://ankurm.com/spring-ai-2-0-tool-calling/) |
|
||||
| [`structured-output/`](structured-output) | `ChatClient.entity()` mapping LLM responses to Java records, lists and maps; `StructuredOutputValidationAdvisor` retrying non-conforming JSON with a real enum-constrained schema, including a captured run that exhausts every retry without throwing. Spring Boot 4.1.1, Spring AI 2.0.1, Java 25. | [Structured Output in Spring AI 2.0](https://ankurm.com/spring-ai-2-0-structured-output/) |
|
||||
| [`ollama-local/`](ollama-local) | Chat and embeddings against a real local `qwen2.5:0.5b`/`all-minilm`, no API key, driven by a Testcontainers-managed Ollama container started from a baked image; a confirmed model unload via `keep_alive: 0` and `/api/ps`, not a scripted model anywhere. Spring Boot 4.1.1, Spring AI 2.0.1, Testcontainers 2.0.5, Java 25. | [Run LLMs Locally with Spring AI and Ollama](https://ankurm.com/spring-ai-2-0-ollama-local/) |
|
||||
|
||||
Upgrading from Spring AI 1.x: [migration guide](https://ankurm.com/spring-ai-1-to-2-migration-guide/).
|
||||
|
||||
Reference in New Issue
Block a user