Top 25 Spring AI and GenAI Interview Questions for Java Developers
25 interview questions on RAG, embeddings, tool calling, MCP, agents, evaluation and cost control for Java developers, with short answers in accordions and runnable proof for the tricky ones.
Interviewers for Java roles that touch generative AI tend to ask the same cluster of questions: how retrieval works, why tool calls fail, how you would stop a bill from exploding. This post collects 25, grouped by topic, each with a short answer you can say out loud. Tap a question to open it. Where an answer can be proven with a few lines of code, I ran the code and show what came out.
Versions and limits. Spring AI 2.0.1 (GA) on Spring Boot 4.1.1, JDK 25. The runnable snippets are in one file, InterviewDemos.java. They use a toy keyword “embedding” and call no model, so they show how the Spring AI pieces behave, not how any real model answers. Where an answer is general GenAI knowledge rather than something I ran, it is labelled “general”. Interview answers also depend on your seniority and your company; treat these as a base to adapt.
Two traps that interviewers love
Both come from running the file above, and both are worth knowing before you walk in.
1. Your tool’s parameter is called arg0. I wrote a @Tool method taking a city and compiled it with plain javac. The schema the model would receive:
Java does not keep parameter names in class files unless you compile with -parameters. The model was told the argument is arg0; a call with {"city":"Mumbai"} then matched nothing, and the method received null. With -parameters the schema says city and the result was 31C in Mumbai. Spring Boot’s Maven parent turns the flag on, so this bites when you build without it, in a plain javac setup or a custom compiler configuration.
2. Similarity search always returns something. Three documents, a query “refund please”, topK(2):
hit: How to request a refund for an invoice score=0.7071067811865475
hit: Reset your password from the login page score=0.0
The second hit has a score of zero and still came back, because top-K returns the K nearest whatever their distance. Without a similarity threshold, a RAG pipeline hands the model irrelevant context when the knowledge base has nothing good to offer.
The map: how the questions connect
The diagram is the order of work in a typical retrieval-based assistant, and the caption says which question numbers belong to which stage.
Fundamentals
1. What problem does Spring AI solve?
It gives Java applications one portable API over chat models, embedding models, vector stores and tools, with Spring conventions: auto-configuration, advisors, observability. You code against ChatClient and VectorStore and swap the provider by configuration. See ChatClient on Spring Boot 4.1. (general)
2. What is the difference between ChatModel and ChatClient?ChatModel is the low-level interface to one provider: prompt in, response out. ChatClient is the fluent layer on top: default system prompts, advisors, tools, structured output. Most application code should use ChatClient. (general)
3. What is an advisor?
An interceptor around a model call, in the way a servlet filter wraps a request. Advisors add memory, retrieval, logging, redaction, token budgets and the tool-calling loop. Order matters because each advisor sees the request on the way in and the response on the way out. Worked examples: Writing custom advisors.
4. How do you get structured data (a Java record) out of an LLM, and what goes wrong?
Ask for a type with entity(...). Spring AI appends a format instruction containing the JSON schema of your class to the prompt, then parses the reply. I printed it for a two-field record: the instruction says to reply with RFC 8259 JSON, no markdown fences, following a draft 2020-12 schema. It parsed {"category":"billing","urgency":2} into Ticket[category=billing, urgency=2]. A truncated reply failed with a StreamReadException. So the failure mode is the model not complying, and your answer is a validation and retry step. See Structured Output in Spring AI 2.0.
Embeddings and vector stores
5. What is an embedding?
A list of numbers that places a piece of text so that similar meanings sit close together. The distance between two vectors is the measure of similarity. (general)
6. What is cosine similarity and what does a score of 0 mean?
Cosine similarity is the angle between two vectors: 1 means the same direction, 0 means unrelated, in the toy demo. In my run “refund” against “refund invoice” scored 0.707 and “refund” against “password” scored 0. Note that Spring AI’s Redis-backed store reports a rescaled score, (1 + cosine) / 2, so the same pair can show different numbers in different stores; I measured that in the semantic caching post.
7. When do you pay for embeddings?
At indexing, once per chunk, and at query time, once per question. In the demo, indexing three documents made 3 embedding calls and one query made a fourth. If you change the embedding model you must re-embed everything, because vectors from different models are not comparable. If you only change the store, you can copy vectors; see VectorStore.upsert() in Spring AI 2.1 (a milestone feature).
8. How do you choose a vector store?
By what you already operate and by recall under your filters, not by benchmark headlines. I compared pgvector, Redis, Qdrant and Elasticsearch on one dataset in Choosing a Vector Store for Spring AI. A good interview answer names metadata filtering and index defaults as the traps.
Retrieval-augmented generation
9. Explain RAG in one minute.
Split your documents into chunks, embed and store them. At question time embed the question, fetch the nearest chunks, put them in the prompt, and ask the model to answer from them. It lets a model use data it was never trained on without retraining. Full pipeline: Production-grade RAG with Spring AI.
10. How does chunk size affect retrieval?
Smaller chunks are more precise and cost more to store; larger chunks carry more context and dilute the match. With TokenTextSplitter on a 200-sentence text, chunk size 100 gave 20 chunks, 400 gave 5, and 800 gave 3. Tune it against real questions, with overlap if answers straddle boundaries.
11. What happens when the knowledge base has no good answer?
See trap 2: top-K still returns something. Set a similarity threshold, and tell the model in the prompt to say it does not know when the context is insufficient. Then test that behaviour. (the threshold is general; the zero-score hit is measured)
12. What is reranking and when is it worth it?
A second, slower model scores the retrieved chunks against the question and reorders them. It is worth it when the first-stage search gets the right chunk into the top 20 but not the top 3. It adds latency and cost per query. The example is in the complete RAG example.
13. How do you reduce hallucination in a RAG system?
Retrieve better (chunking, filters, threshold), instruct the model to answer only from context and to cite, validate the answer against the sources, and measure with a labelled test set. Prevention is probabilistic, so you also need evaluation (questions 20 and 21). (general)
Tool calling and MCP
14. How does tool calling work?
You describe a function (name, description, JSON schema). The model replies with a request to call it with arguments. Your code runs it and sends the result back, and the model continues. In Spring AI 2.0 the advisor layer runs that loop. The model never executes anything. Details: Tool Calling in Spring AI 2.0. Mention trap 1 here.
15. What is in the schema the model sees for a @Tool method?
The tool name (the method name by default), your description, and a JSON schema with each parameter’s name, type and @ToolParam description, marked required, with additional properties disallowed. That is the entire contract, so the quality of those descriptions decides whether the model calls the tool correctly.
16. What is MCP and how is it different from a @Tool?
The Model Context Protocol is a standard for exposing tools, resources and prompts from a separate server that any compatible client can use. A @Tool lives inside your JVM; an MCP tool lives behind a protocol. Spring AI 2.0’s MCP server defaults to Streamable HTTP. Build one in MCP server, consume one in MCP client.
17. How do you secure an MCP server?
Treat it like any API: authenticate with OAuth2, authorise per tool, and log every call including denials. Do not rely on the model to behave. See Securing an MCP Server with Spring Security 7.
Agents
18. What is an agent, compared with a plain LLM call?
An agent loops: it decides, calls tools, reads results and decides again until a goal is met. That makes cost, latency and failure modes unbounded unless you cap steps. A plain call is one request and one answer. (general) Two Java approaches: Embabel and, for packaging know-how, Agent Skills in Spring AI.
19. How do you give an agent memory?
Chat memory stores prior messages per conversation id and an advisor replays them. Keep a window of recent messages so the prompt does not grow forever. Know what your repository stores: in 2.1.0-M1 I found JDBC memory keeps only message type and text, so reasoning parts are lost (Spring AI 2.1 message parts). Full coverage: Chat Memory in Spring AI 2.0.
Evaluation and testing
20. How do you test code whose output is non-deterministic?
Split it. Test the deterministic parts (prompt building, parsing, tool code, routing) with a scripted model in ordinary unit tests. Test model quality separately with a labelled dataset and a pass-rate gate. See Testing LLM apps in Java.
21. What is LLM-as-judge and what is its weakness?
A second model grades the first model’s answer against criteria, for example relevancy or factual support. Its weakness is that the judge is also a model: it can be noisy and biased. Calibrate it on cases you labelled by hand, and run it several times when the decision matters. (general; evaluator classes in the testing post)
Cost, safety and operations
22. How do you control LLM cost?
Measure first (token usage per endpoint, see Observability for Spring AI), then shrink prompts, cache repeated questions (semantic caching), route easy requests to a cheaper model, and cap per-user budgets in an advisor.
23. What is prompt injection and how do you defend against it?
Text in a document, tool result or user message that tries to override your instructions. Defence is layered: filter retrieved content, allow-list tools and arguments, and validate output before acting on it. No single layer is enough. See Prompt Injection Defense in Spring AI.
24. How do you avoid vendor lock-in?
Code to ChatClient, keep provider-specific options in configuration, and have a gateway layer for failover and budgets. Know where the portable API ends: provider options differ. See switching providers and the LLM gateway pattern.
25. Can you run it without sending data to a third party?
Yes, with a local model for chat and embeddings, at the cost of quality and hardware. Spring AI supports Ollama; see Run LLMs locally with Spring AI and Ollama.
Run the demos yourself
Create a Maven project with the Spring Boot 4.1.1 BOM and the Spring AI 2.0.1 BOM, and add spring-ai-client-chat and spring-ai-vector-store. Compile InterviewDemos.java with javac -parameters to see the fixed tool schema, and without the flag to see arg0. No network or key is needed.
Should you even do this? Memorising answers is the weak way to use this list. Interviewers for these roles tend to ask a follow-up that starts with “and what went wrong when you tried it?”. Pick three questions, run the code for them yourself, and be ready to describe a failure. Also expect questions I did not cover, such as fine-tuning or model internals, which this Spring-focused list leaves out.
No Comments yet!