GraphRAG in Java with Neo4j and Spring AI: Turning Documents into a Knowledge Graph
Build GraphRAG in Java: extract entities and relations with Spring AI structured output, store them in Neo4j with a vector index, answer multi-hop questions with Cypher, and measure it against plain vector RAG on the same 35 documents. The extraction step is a rule-based stand-in, not an LLM, and the post says so.
Ask a vector-search RAG system “Who does the leader of the team that owns the Payments Gateway report to?” and it does what it always does: it finds the passages that look like the question. The one about the Payments Gateway, yes. The one that says who leads the Payments Team, maybe. The one that says who that person reports to? It never mentions the Payments Gateway, the team or the word “leader”, so it sits wherever its resemblance to the question puts it, which may be below passages with no answer in them at all.
That is a multi-hop question: the answer is assembled from facts in three different documents, and each fact is only reachable through a name found in the previous one. GraphRAG is the family of techniques that handle this by turning the documents into a knowledge graph first — entities as nodes, the relations between them as edges — and answering by walking the graph instead of ranking passages. Microsoft’s 2024 paper made the term famous; almost every example of it is Python. This post builds the Java version end to end with Spring AI 2.0 and Neo4j, runs it, and measures it against plain vector RAG on the same documents.
Versions this was built and run against. Spring Boot 4.1.2 and Spring AI 2.0.2, both the newest GA releases on Maven Central (published 22 October 2026; Spring AI 2.1.0-RC1 and Spring Boot 4.2.0-RC1 are release candidates, not used). Neo4j 2026.10.1-community in Docker (the image tag was pushed 23 October 2026; Neo4j now numbers releases by year and month), and the same tests again on 5.26.32-community. Neo4j Java driver 6.1.0 (from the Spring Boot BOM), Testcontainers 2.0.5, Temurin JDK 25.0.5+8, and all-MiniLM-L6-v2 embeddings running inside the JVM.
What is real here and what is a stand-in. No language model was called anywhere in this project. The extraction step (read a document, list its entities and relations) runs through the real Spring AI ChatClient.entity(...), the real JSON schema and the real parser, but the “model” behind it is RuleBasedExtractorModel: a class that spots eight fixed sentence patterns with regular expressions and replies with the JSON a good model would. It is deterministic, so every number below repeats, and it says nothing about how well a real model extracts entities from messy prose. Neo4j, both vector indexes, the embeddings, the Cypher and every retrieval number are real. The final step, a model writing the answer, is not run at all; the project prints the prompt it would send. Every block marked as output below was produced by the test suite and sits in the repository.
A multi-hop question is made of facts that do not look like the question
The test bed is a small, invented company: 35 short documents. Twenty-one of them state facts (“The Payments Gateway is owned by the Payments Team”, “The Payments Team is led by Priya Raman”, “Priya Raman reports to Dana Okafor”) and fourteen are distractors that reuse the same vocabulary: a Payments Team offsite, a card-data policy, a checkout latency dashboard. They are all in src/main/resources/corpus. Nine questions, written by hand after the corpus, range from one hop (“Who leads the Fulfilment Team?”) to three.
Take the question from the top of the page. Three documents hold the answer, chained by names.
Read the diagram left to right. The question shares words with the first document only. The second is reachable because the first names the Payments Team; the third because the second names Priya Raman. A vector search never follows a name: it embeds the question once and returns whatever sits closest. Whether the third document makes the cut is luck, and distractor documents about payments compete for the same few slots.
“Closest” here means cosine similarity between embedding vectors — lists of 384 numbers that a model produces for a piece of text so that texts about similar things get similar lists. If that is new, the RAG ingestion post builds the same embedding step from scratch, and the production RAG post shows the plain pipeline this one is compared against.
Going deeper: why top-k cannot simply be raised
Raising k does help, and the results below show it: top 8 completes more questions than top 4. But the context grows with k and so does the noise the model must read past, and a chain of four or five hops needs a k that depends on the question. The graph approach changes what is retrieved (the facts on a path) rather than how many passages are retrieved. Two other fixes for weak ranking are covered in hybrid search and cross-encoder reranking; they improve the ranking of passages but still return passages, so a hop whose passage shares nothing with the question stays out of reach.
A knowledge graph is nodes, typed edges, and a memory of where each edge came from
A property graph, the model Neo4j uses, has two ingredients. Nodes are things, each with a label and properties: a Service, a Team, a person. Relationships connect two nodes, have a type and a direction, and can carry properties too. Neo4j’s query language, Cypher, draws the pattern you want in ASCII art: (g)-[:OWNED_BY]->(t) means “a node g with an OWNED_BY relationship pointing at a node t”. The rest of the language is mostly filters and what to return.
Here is a slice of the graph this project builds from the 35 documents.
Follow the amber and green nodes: Payments Gateway, OWNED_BY, Payments Team, LED_BY, Priya Raman, REPORTS_TO, Dana Okafor. That is the whole multi-hop question answered by a walk of three relationships, with nothing to rank. Two details matter later. Entities have types (system, team, person, vendor, incident), which is what lets two things with similar names be told apart. And each edge stores the ids of the chunks that stated it, so the graph never replaces the text: it points back at it, and the final prompt contains the original sentences.
Going deeper: what Microsoft’s GraphRAG does that this project does not
The paper is “From Local to Global: A Graph RAG Approach to Query-Focused Summarization” (Edge et al., April 2024). Its method, in the authors’ words: a language model builds an entity knowledge graph from the documents, groups closely related entities into clusters, and writes a summary of each cluster in advance. To answer, it drafts partial responses from the cluster summaries and merges them. That design targets global questions such as “what are the main themes across this corpus?”, which no single passage or path can answer. This project implements only the part before that: extraction into a graph and graph-guided retrieval for specific multi-hop questions. It has no clustering and no pre-written summaries, so do not read the numbers below as a measurement of Microsoft’s system. The GraphRAG project site is the reference implementation.
Step 1: ask the model to fill in a Java record
The pipeline has two halves, drawn below. At indexing time each document goes to a language model with one instruction: list the entities and the relations you find. At question time no model is involved until the very end.
The top row is the work that happens once per document, and it is where a language model earns its cost. The bottom row repeats for every question and uses only the embedding model and Cypher. The dashed boxes are the two places a language model belongs; in this project neither one runs a real model.
The extraction contract is a Java record. Spring AI turns it into a JSON schema, appends that schema to the prompt, and parses the reply back into the record (the structured output post covers entity() in depth). The two enums are the important design choice: they close the vocabulary, so the model cannot invent a “works_closely_with” relation or a “Concept” node type. Source: src/main/java/com/ankurm/graphrag/Extraction.java.
public record Extraction(List<Entity> entities, List<Relation> relations) {
public record Entity(String name, EntityType type) {
}
public record Relation(String source, RelationType type, String target) {
}
public static Extraction empty() {
return new Extraction(List.of(), List.of());
}
}
public Result extract(Corpus.Doc doc) {
Extraction raw = client.prompt()
.user(u -> u.text(USER).param("document", doc.text()))
.call()
.entity(Extraction.class);
return clean(raw);
}
Here is what the model receives for one document and what comes back, from output/01-extraction.txt. The reply was produced by the stand-in; the prompt, the schema and the parsed record are exactly what a real model would see and what your code would receive.
entity SYSTEM Checkout Service
entity SYSTEM Payments Gateway
entity SYSTEM Inventory Service
entity TEAM Storefront Team
relation Checkout Service -[DEPENDS_ON]-> Payments Gateway
relation Checkout Service -[DEPENDS_ON]-> Inventory Service
relation Checkout Service -[OWNED_BY]-> Storefront Team
Across all 35 chunks, one call each:
model calls : 35 (one per chunk)
characters sent : 72437 (about 18109 tokens at 4 characters per token, an estimate)
entity mentions (before merging): 54
relations : 33
chunks with no relation at all : 14
Indexing cost therefore scales with the corpus: one model call per chunk, here about 18,000 input tokens by the four-characters-per-token rule of thumb (an estimate, not a tokenizer count). Question cost does not change. Fourteen chunks produced no relation at all, which is correct: they are the distractors.
Show the full prompt the model receives, JSON schema included
The block is the whole user message for the Checkout Service document, verbatim from output/01-extraction.txt. The system message (the instruction to use names exactly as written and never to infer a relation) is Extractor.java. Notice the enum lists inside the schema: they are how the closed vocabulary reaches the model.
Extract the entities and relations from the document.
DOCUMENT_START
Checkout Service. The Checkout Service takes a customer's cart and turns it into an order. It depends on the Payments Gateway to take card payments and on the Inventory Service to reserve stock. The Checkout Service is owned by the Storefront Team. Its latency dashboard lives in the shared Grafana folder.
DOCUMENT_END
Your response should be in JSON format.
Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
Do not include markdown code blocks in your response.
Remove the ```json markdown from the output.
Here is the JSON Schema instance your output must adhere to:
```{
"$schema" : "https://json-schema.org/draft/2020-12/schema",
"type" : "object",
"properties" : {
"entities" : {
"type" : "array",
"items" : {
"type" : "object",
"properties" : {
"name" : {
"type" : "string"
},
"type" : {
"type" : "string",
"enum" : [ "SYSTEM", "TEAM", "PERSON", "VENDOR", "INCIDENT", "PLACE" ]
}
},
"required" : [ "name", "type" ],
"additionalProperties" : false
}
},
"relations" : {
"type" : "array",
"items" : {
"type" : "object",
"properties" : {
"source" : {
"type" : "string"
},
"target" : {
"type" : "string"
},
"type" : {
"type" : "string",
"enum" : [ "DEPENDS_ON", "OWNED_BY", "LED_BY", "HOSTED_ON", "AFFECTED", "CAUSED_BY", "REPORTS_TO", "LOCATED_IN" ]
}
},
"required" : [ "source", "target", "type" ],
"additionalProperties" : false
}
}
},
"required" : [ "entities", "relations" ],
"additionalProperties" : false
}```
The happy path above took a paragraph; the failures are where indexing jobs die at document 12,000. The test ExtractionTest makes the stand-in misbehave in six ways that real models do, and records what Spring AI 2.0.2 does with each (output/02-extraction-failures.txt).
--- JSON wrapped in a markdown fence (reply starts with the fence) ---
parsed OK, 2 relations
--- the same fence, after one sentence of chat ---
tools.jackson.core.exc.StreamReadException: Unrecognized token 'Here': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
--- JSON cut off two thirds of the way through ---
tools.jackson.core.exc.UnexpectedEndOfInputException: Unexpected end-of-input in property name
--- a relation type outside the allowed list (IS_RESPONSIBLE_FOR) ---
tools.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `com.ankurm.graphrag.RelationType` from String "IS_RESPONSIBLE_FOR": not one of the values accepted for Enum class: [OWNED_BY, LOCATED_IN, AFFECTED, HOSTED_ON, LED_BY, DEPENDS_ON, REPORTS...
--- a relation type that is a Cypher fragment ---
tools.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `com.ankurm.graphrag.RelationType` from String "OWNED_BY]->(x) DETACH DELETE x //": not one of the values accepted for Enum class: [OWNED_BY, LOCATED_IN, AFFECTED, HOSTED_ON, LED_BY, DEPE...
--- a relation to an entity that was never listed ---
relations kept 2, dropped 1
Reading it top to bottom. A reply that starts with a markdown fence parses fine: Spring AI strips it. The same fence after one friendly sentence does not, and the exception is a Jackson StreamReadException, not anything that mentions extraction. A truncated reply (a model that hit its output limit) fails the same way. A relation type outside the enum is rejected with the list of accepted values. A relation to an entity never listed passes parsing, because it is valid JSON of the right shape; only the cleaning step in Extractor.clean catches it, and without that step the graph writer would run MATCH on a node that does not exist and silently create no edge.
The fifth case is a security boundary, not a parsing nicety. Cypher has no parameter for a relationship type, so writing an edge means putting the type into the query text. The model’s output is untrusted text (a poisoned document can influence it), and the stand-in’s reply here is a relation type of OWNED_BY]->(x) DETACH DELETE x //. Because type is an enum in the record, that string never gets as far as the query: deserialisation fails first. The query builder then uses type.name() of an enum constant, so only eight literals can ever be concatenated. Make the type a plain String “for flexibility” and you have built a Cypher injection hole fed by an LLM. The same reasoning drives the guards in the text-to-SQL post.
byType.forEach((type, rows) -> run("""
UNWIND $rows AS row
MATCH (a:Entity {key: row.from}), (b:Entity {key: row.to})
MERGE (a)-[r:%s]->(b)
SET r.sources = CASE WHEN row.chunk IN coalesce(r.sources, []) THEN r.sources
ELSE coalesce(r.sources, []) + row.chunk END
""".formatted(type.name()), Map.of("rows", rows)));
That is the write path (src/main/java/com/ankurm/graphrag/GraphStore.java): one UNWIND per relationship type, and a sources list on each edge that accumulates the id of every chunk that stated it. Exceptions at the first three failures are what you retry; the StructuredOutputValidationAdvisor from the structured output post re-asks the model with the error in the prompt.
Going deeper: the allow-list enum and the two things it cannot guard
public enum RelationType {
DEPENDS_ON("depends on"),
OWNED_BY("is owned by"),
LED_BY("is led by"),
HOSTED_ON("is hosted on"),
AFFECTED("affected"),
CAUSED_BY("was caused by"),
REPORTS_TO("reports to"),
LOCATED_IN("is located in");
// ...phrase() and a constructor
The enum guarantees the shape of what reaches Cypher. It cannot guarantee the truth: a model can still write DEPENDS_ON where the document said the opposite, or invent a relation the text only hints at. That is why the system prompt says “List a relation only when the document states it” and why every edge keeps its source chunk ids — a wrong edge can be traced to the sentence that produced it. Second, the schema allows one relation type per pair of entities per sentence; a document saying a team both owns and operates a service needs two relations, and an enum that is too small silently loses the second.
Step 2: the same thing under two names
The model reads each document separately, so it cannot know that INC-2060’s Card Gateway and the service page’s Payments Gateway are one system. (INC-2060’s own text says so: “The Card Gateway is the on-call channel’s name for the Payments Gateway”. A model reading only that chunk could emit the alias; the stand-in does not.) If both names become nodes, the incident is attached to a node that nothing else touches. Output from output/03-graph-build.txt, building the graph with exact-name matching only:
--- A. exact-name matching only (no alias table, similarity merge off) ---
entities: 28
Payments Gateway : -[DEPENDS_ON]-> Ledger Service; -[HOSTED_ON]-> Arcadia Cloud; -[OWNED_BY]-> Payments Team; <-[DEPENDS_ON]- Checkout Service
Card Gateway : <-[AFFECTED]- INC-2060
Twenty-eight entities, and the Payments Gateway has no incident. A question like “which incidents affected services the Checkout Service depends on?” will miss INC-2060. This is the central maintenance problem of GraphRAG: entity resolution, deciding which mentions are the same thing.
The tempting fix is to merge names whose embeddings are close. I tried it, and it is worth seeing the numbers (same file):
--- same-type name pairs by cosine of their embeddings, top 8 of 72 ---
0.806 INCIDENT INC-2057 <-> INC-2060
0.797 INCIDENT INC-2041 <-> INC-2057
0.758 INCIDENT INC-2041 <-> INC-2060
0.681 SYSTEM Card Gateway <-> Payments Gateway
0.540 SYSTEM Ledger Service <-> Inventory Service
0.535 TEAM Storefront Team <-> Platform Team
0.530 TEAM Core Data Team <-> Platform Team
0.519 SYSTEM Inventory Service <-> Checkout Service
-- Card Gateway <-> Payments Gateway: cosine 0.681, rank 4
The number line shows the problem. The pair that should merge scores 0.681, which is below three pairs of different incidents (“INC-2057” and “INC-2060” look alike to a text embedding because they are the same shape of string). There is no threshold that merges the alias without merging incidents first:
--- what a similarity-only merge would do at each threshold ---
threshold 0.90 -> 0 merges:
threshold 0.80 -> 1 merges: INC-2057 + INC-2060
threshold 0.70 -> 3 merges: INC-2057 + INC-2060; INC-2041 + INC-2057; INC-2041 + INC-2060
threshold 0.60 -> 4 merges: INC-2057 + INC-2060; INC-2041 + INC-2057; INC-2041 + INC-2060; Card Gateway + Payments Gateway
Do not merge entities by embedding similarity alone. At 0.80 the first merge is INC-2057 with INC-2060 — two different outages — and the true alias is not merged until 0.60, after all three wrong incident merges. Identifier-like names (incident numbers, ticket ids, error codes) are close to each other in embedding space precisely because they look alike. The project’s EntityResolver keeps the similarity merge as an option and defaults it off. What fixed the graph here is a one-line alias table; in a real system you would generate candidate pairs by similarity and have a person or a second model call confirm them.
With the alias Card Gateway = Payments Gateway (GraphFixture.ALIASES), the graph is correct:
--- B. plus a one-line alias table (Card Gateway = Payments Gateway) ---
entities: 27
Payments Gateway : -[DEPENDS_ON]-> Ledger Service; -[HOSTED_ON]-> Arcadia Cloud; -[OWNED_BY]-> Payments Team; <-[AFFECTED]- INC-2060; <-[DEPENDS_ON]- Checkout Service
chunks: 35, MENTIONS edges: 54, dropped relations: 0
--- relationships by type ---
DEPENDS_ON 6
OWNED_BY 6
HOSTED_ON 5
LED_BY 5
REPORTS_TO 4
AFFECTED 3
CAUSED_BY 2
LOCATED_IN 2
Twenty-seven entities, 54 mentions linking chunks to entities, and 33 relationships whose types add up to the count in the table. INC-2060 now hangs off the Payments Gateway.
Step 3: put it in Neo4j, with Testcontainers and two vector indexes
Neo4j needs a real server, so the tests start one with Testcontainers. Two dependency details cost me a run each. The artifact is testcontainers-neo4j in Testcontainers 2.0.5 and the class is org.testcontainers.neo4j.Neo4jContainer. And the first new Neo4jContainer(...) failed with NoClassDefFoundError: org/apache/commons/lang3/exception/ExceptionUtils until I added commons-lang3 as a test dependency, which Testcontainers 2.x needs but does not pull in. Both are in pom.xml:
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-neo4j</artifactId>
<scope>test</scope>
</dependency>
<!-- Testcontainers 2.x reads its configuration through commons-lang3 but does not bring it in: without this
the first `new Neo4jContainer(...)` fails with NoClassDefFoundError. -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<scope>test</scope>
</dependency>
One container serves the whole test run (src/test/java/com/ankurm/graphrag/support/GraphFixture.java). The Community Edition image needs no licence acceptance; a fresh container took about a minute to accept connections on this 2-vCPU sandbox. This machine had Docker installed but no running daemon, so I started dockerd by hand before the first run; on a laptop with Docker Desktop that step does not exist.
container = new Neo4jContainer(IMAGE).withAdminPassword("graphrag-demo");
container.start();
driver = GraphDatabase.driver(container.getBoltUrl(), AuthTokens.basic("neo4j", container.getAdminPassword()));
pipeline = new Pipeline(driver, MiniLm.get(), STUB);
pipeline.loadChunks(DOCS);
public void initSchema() {
run("CREATE CONSTRAINT entity_key IF NOT EXISTS FOR (e:Entity) REQUIRE e.key IS UNIQUE", Map.of());
run("CREATE VECTOR INDEX " + ENTITY_INDEX + " IF NOT EXISTS FOR (e:Entity) ON (e.embedding) "
+ "OPTIONS {indexConfig: {`vector.dimensions`: " + embeddings.dimensions()
+ ", `vector.similarity_function`: 'cosine'}}", Map.of());
run("CALL db.awaitIndexes(60)", Map.of());
}
image : neo4j:2026.10.1-community
server : Neo4j Kernel 2026.10.1 community
--- vector indexes (one from Neo4jVectorStore for chunks, one of ours for entity names) ---
chunk_embedding on (:Chunk).embedding ONLINE
entity_name_embedding on (:Entity).embedding ONLINE
A vector index is what lets Neo4j answer “which nodes have an embedding closest to this one?” with db.index.vector.queryNodes, the same operation a vector database offers, now inside the graph database. That is the useful property: one system holds the vectors and the relationships, so a single Cypher statement can find a starting node by similarity and walk away from it. Section “When you know the shape” shows one.
Going deeper: Neo4j versions, the vector index and the chunk store
The project’s tests also ran unchanged on neo4j:5-community, which resolved to Neo4j Kernel 5.26.32 (transcript: output/03b-graph-build-neo4j-5.txt); the vector index statements are identical. The version header from that run:
image : neo4j:5-community
server : Neo4j Kernel 5.26.32 community
Spring AI’s Neo4jVectorStore stores each document as a node with the label you give it, writes the embedding with db.create.setNodeVectorProperty, and creates its index with CREATE VECTOR INDEX ... IF NOT EXISTS. Our entity index uses the same procedure and a cosine similarity function. The builder call:
A third kind of edge, (:Chunk)-[:MENTIONS]->(:Entity), records which chunk mentioned which entity (54 of them above). The path walk below deliberately excludes it by listing only the eight relation types, otherwise every walk would hop through chunks and reach every entity mentioned in the same document.
Step 4: find where to start, walk the graph, keep the paths that match
Retrieval has three moves, all in src/main/java/com/ankurm/graphrag/GraphRetriever.java and none involving a language model. Link: find the entities the question is about. Walk: from each, collect every path of up to N relationships. Rank: turn each path into a sentence, embed it, and keep the few closest to the question.
Linking first. Without a model to pick out names, the code cuts the question into every span of one to four words (skipping spans that start or end with a stop word), embeds all of them in one batch, and asks the entity vector index for the nearest entity to each. A span counts if its cosine clears a threshold, 0.80 here:
public List<Seed> link(String question) {
String[] words = question.replaceAll("[?,.]", "").split("\\s+");
List<String> spans = new ArrayList<>();
for (int i = 0; i < words.length; i++) {
for (int n = 1; n <= 4 && i + n <= words.length; n++) {
String first = words[i].toLowerCase();
String last = words[i + n - 1].toLowerCase();
if (STOP.contains(first) || STOP.contains(last)) {
continue;
}
spans.add(String.join(" ", java.util.Arrays.copyOfRange(words, i, i + n)));
}
}
List<float[]> vectors = embeddings.embed(spans);
Map<String, Seed> best = new LinkedHashMap<>();
for (int i = 0; i < spans.size(); i++) {
for (GraphStore.EntityHit hit : graph.nearestEntities(vectors.get(i), 1)) {
if (hit.score() >= linkThreshold) {
Seed seed = new Seed(spans.get(i), hit.name(), hit.score());
best.merge(hit.name(), seed, (a, b) -> a.score() >= b.score() ? a : b);
}
}
}
return best.values().stream().sorted(Comparator.comparingDouble(Seed::score).reversed()).toList();
Q2 Who leads the team that owns the Payments Gateway?
'Payments Gateway' -> Payments Gateway (1.00)
'Payments' -> Payments Team (0.88)
'team' -> Fulfilment Team (0.83)
Q5 Where is the company that hosts the database behind the Ledger Service located?
'Ledger Service' -> Ledger Service (1.00)
'database' -> Orders Database (0.82)
Q7 Which services stop working if the Orders Database goes down?
'Orders Database' -> Orders Database (1.00)
Linking by similarity finds noise as well as names. For Q2 the span “team” resolved to the Fulfilment Team at 0.83, a team the question never mentions, and “Payments” to the Payments Team at 0.88. For Q5 the span “database” found the Orders Database at 0.82, which happened to be right. I fixed 0.80 before running any evaluation. The sweep later in the post shows 0.85 and above do better on these nine questions, but that is a result on the very questions I would be tuning against, so treat it as a direction, not a setting to copy.
Walking is one Cypher query per seed: every simple path (no node twice) of one to N relationships, in either direction, over the eight allowed types (src/main/java/com/ankurm/graphrag/GraphStore.java):
public List<GraphPath> paths(String seedKey, int maxHops, int limit) {
String types = Arrays.stream(RelationType.values()).map(Enum::name).collect(Collectors.joining("|"));
List<Record> rows = query("""
MATCH p = (s:Entity {key: $key})-[:%s*1..%d]-(t:Entity)
WHERE all(n IN nodes(p) WHERE single(m IN nodes(p) WHERE m = n))
RETURN [r IN relationships(p) | {from: startNode(r).name, type: type(r), to: endNode(r).name,
sources: r.sources}] AS edges
LIMIT $limit
""".formatted(types, maxHops), Map.of("key", seedKey, "limit", limit));
List<GraphPath> out = new ArrayList<>();
for (Record row : rows) {
List<Edge> edges = new ArrayList<>();
for (Value e : row.get("edges").values()) {
edges.add(new Edge(e.get("from").asString(), RelationType.valueOf(e.get("type").asString()),
e.get("to").asString(), e.get("sources").asList(Value::asString)));
}
out.add(new GraphPath(seedKey, edges));
}
return out;
The hop limit is the knob with the sharpest edge. Candidate paths from three seeds, by hop limit (same transcript):
The count roughly doubles with every hop after the first. On 27 entities that is harmless; on a graph with a hub node (a shared database, a company-wide team) it explodes, which is why the third move exists. Each candidate path is rendered as a sentence (“Payments Gateway is owned by Payments Team; Payments Team is led by Priya Raman”), embedded, and scored by cosine against the question; the best few survive, and the context is the set of chunks that stated their edges (retrieve()):
public Retrieval retrieve(String question, int depth, int topPaths) {
List<Seed> seeds = link(question);
// one walk per seed; a path found from two seeds in opposite directions counts once
Map<Set<String>, GraphStore.GraphPath> unique = new LinkedHashMap<>();
for (Seed s : seeds) {
for (GraphStore.GraphPath p : graph.paths(EntityResolver.key(s.name()), depth, 5000)) {
Set<String> edgeSet = p.edges().stream().map(GraphStore.Edge::sentence).collect(Collectors.toSet());
unique.putIfAbsent(edgeSet, p);
}
}
List<GraphStore.GraphPath> candidates = new ArrayList<>(unique.values());
List<RankedPath> ranked = new ArrayList<>();
if (!candidates.isEmpty()) {
float[] q = embeddings.embed(question);
List<float[]> vectors = embeddings.embed(candidates.stream().map(GraphStore.GraphPath::text).toList());
for (int i = 0; i < candidates.size(); i++) {
ranked.add(new RankedPath(candidates.get(i), EntityResolver.cosine(q, vectors.get(i))));
}
ranked.sort(Comparator.comparingDouble(RankedPath::score).reversed());
}
List<RankedPath> selected = ranked.subList(0, Math.min(topPaths, ranked.size()));
Set<String> chunks = new LinkedHashSet<>();
selected.forEach(rp -> rp.path().edges().forEach(e -> chunks.addAll(e.sources())));
return new Retrieval(question, seeds, candidates.size(), List.copyOf(selected), chunks);
Going deeper: why rank paths at all, and what this ranking gets wrong
Unranked expansion is the simplest GraphRAG: take everything within N hops of the seed and hand it to the model. It works at 27 entities and fails in production for the reason in the bar chart: the neighbourhood is mostly irrelevant, and the model pays for every token of it. Ranking by similarity of the path sentence to the question is cheap and needs no model, but it is a heuristic with two visible weaknesses in the results below: it favours short paths that mention the question’s words over a longer path that is the actual chain, and for “what depends on X” questions the right path reads as a list of dependencies that do not resemble the question at all (Q7). Alternatives: let a model pick relation types from the schema, write the Cypher (text-to-Cypher), or score paths by weights you learn. The last section shows the case where writing the Cypher yourself is simply better.
The comparison: plain vector RAG against the graph, same documents, same questions
Both retrievers see the same 35 chunks in the same Neo4j. Vector RAG is Neo4jVectorStore.similaritySearch with the top 4 or the top 8 chunks. GraphRAG uses the defaults fixed in advance: link spans at cosine 0.80, walk up to 3 hops, keep the 6 best paths, take the chunks that stated their edges. The measure is deliberately modest: are all the documents needed to answer the question in the context? It does not say a model would then answer correctly, only that the evidence was handed over. Per question, from output/04-vector-vs-graph.txt:
id hops | vector k4 vector k8 | graph chunks | question
Q1 1 | yes yes | yes 6 | Who leads the Fulfilment Team?
Q2 2 | yes yes | yes 5 | Who leads the team that owns the Payments Gateway?
Q3 2 | NO NO | yes 4 | Which team owns the root cause of INC-2041?
Q4 3 | NO yes | yes 5 | Who does the leader of the team that owns the Payments Gateway report to?
Q5 3 | NO yes | yes 4 | Where is the company that hosts the database behind the Ledger Service located?
Q6 3 | NO yes | NO 5 | Who leads the teams that own the services the Checkout Service depends on?
Q7 3 | NO NO | NO 5 | Which services stop working if the Orders Database goes down?
Q8 2 | NO NO | yes 6 | Which incidents affected services that the Checkout Service depends on directly?
Q9 3 | NO NO | yes 4 | Who leads the team that owns the service INC-2057 affected?
complete evidence: vector k=4 2/9, vector k=8 5/9, graph 7/9 (graph context averaged 4.9 chunks; vector k=8 is always 8)
1-hop questions (1): vector k=4 1, vector k=8 1, graph 1
2-hop questions (3): vector k=4 1, vector k=8 1, graph 3
3-hop questions (5): vector k=4 0, vector k=8 3, graph 3
The graph retriever completes 7 of 9 against 5 of 9 for top-8 vector search and 2 of 9 for top-4. The pattern is the interesting part. On the one-hop question everything works. On the two-hop questions the graph completes all three and vector search one (Q3, “which team owns the root cause of INC-2041?”, needs the Orders Database page, which a search for the incident does not surface; Q8 needs the alias to be resolved first). On three hops top-8 vector search catches up on three of five by sheer width: eight chunks is a lot of a 35-chunk corpus, a flattering ratio that disappears at scale. Where the graph used 4 to 6 chunks, vector search needed 8.
The graph does not win everywhere, and the two losses are instructive. Q7 (“which services stop working if the Orders Database goes down?”) fails for both. Q6 fails for the graph alone, which missed the Payments Team page: with only 6 paths kept, the path through the Payments Team’s leader never made the cut. The next section is about Q7; the table below is about Q6.
Show the settings sweep and what each retriever missed
The same nine questions with the retriever’s own settings varied (from output/05-graph-sweeps.txt). The cell format is questions completed out of 9, then the average number of chunks in the context. Caution: this sweep ran on the evaluation questions after I had seen the first results, so its best cell is tuned on the test set. It shows direction and cost, not a recommended setting.
Three hops with 20 paths completes all nine, at an average of 10.3 chunks — more context than top-8 vector search. At one hop the graph cannot beat 3 of 9 whatever is kept, because the questions need paths of length two or three. Raising the link threshold from 0.80 to 0.85 gained one question on its own:
With 35 chunks and 9 questions, a difference of one question is within what a different phrasing of the questions could change. Nothing here supports a claim about a larger corpus.
When you know the shape of the question, write the Cypher
Q7 asks for everything that depends, directly or through other services, on the Orders Database. That is not a retrieval problem; it is a graph query with a known shape, and Cypher answers it exactly with a variable-length pattern (output from output/07-fixed-shape-cypher.txt):
--- Q7 Which services stop working if the Orders Database goes down? ---
MATCH (db:Entity {name: $name})<-[:DEPENDS_ON*1..5]-(s:Entity)
RETURN DISTINCT s.name AS service ORDER BY service
-> Checkout Service
-> Ledger Service
-> Payments Gateway
Three services, the same three the question’s evidence set lists. The path-ranking heuristic missed this because the best-matching sentences were about the database’s own hosting and ownership. Q6 has a fixed shape too: dependency, owner, leader.
--- Q6 Who leads the teams that own the services the Checkout Service depends on? ---
MATCH (c:Entity {name: $name})-[:DEPENDS_ON]->(d:Entity)-[:OWNED_BY]->(team:Entity)-[:LED_BY]->(lead:Entity)
RETURN d.name AS dependency, team.name AS team, lead.name AS lead ORDER BY dependency
-> Inventory Service / Fulfilment Team / Tomasz Nowak
-> Payments Gateway / Payments Team / Priya Raman
And the two halves of GraphRAG fit in a single statement when you want them to. This one finds the seed through the entity vector index, then walks from it:
--- Q8 entity linking and the traversal in ONE statement: the vector index finds the seed, MATCH walks from it ---
CALL db.index.vector.queryNodes('entity_name_embedding', 1, $vector) YIELD node AS c, score
MATCH (c)-[:DEPENDS_ON]->(d:Entity)<-[:AFFECTED]-(i:Entity)
RETURN c.name AS seed, score, d.name AS dependency, i.name AS incident ORDER BY incident
$vector = embedding of the span "Checkout Service"
-> seed Checkout Service (1.00): INC-2057 affected Inventory Service
-> seed Checkout Service (1.00): INC-2060 affected Payments Gateway
A model-written version of these queries is text-to-Cypher, and it needs the same guards as text-to-SQL. I did not run one: no model was available. If you let a model write the Cypher, give it a read-only database user, check the statement before running it (one statement, no CREATE, MERGE, DELETE or CALL outside an allow-list), cap the rows and the time. The text-to-SQL post works through exactly that set of defences for SQL. For a handful of recurring question shapes, hand-written parameterised queries behind a router are safer and cheaper than any generated Cypher.
What the model finally sees
The last step is ordinary RAG: put the evidence in front of a model and ask. The graph contributes the evidence in two forms — the selected paths as short fact lines, and the source passages that stated them. For Q4 the prompt the project builds (output/06-prompt.txt; no model was called, nothing here is an answer) opens like this:
QUESTION: Who does the leader of the team that owns the Payments Gateway report to?
FACTS FROM THE GRAPH:
- Payments Gateway is owned by Payments Team
- Payments Gateway is owned by Payments Team; Payments Team is led by Priya Raman; Priya Raman reports to Dana Okafor
- Payments Gateway is owned by Payments Team; Payments Team is led by Priya Raman
- Payments Gateway is owned by Payments Team; INC-2060 affected Payments Gateway
- Payments Gateway is owned by Payments Team; Checkout Service depends on Payments Gateway
- Payments Gateway is owned by Payments Team; Payments Gateway is hosted on Arcadia Cloud
The fact lines already contain the chain: owned by Payments Team; led by Priya Raman; reports to Dana Okafor. The passages follow, so the model can verify the facts against the original text and cite them. The code that assembles this is src/main/java/com/ankurm/graphrag/Answerer.java:
QUESTION: Who does the leader of the team that owns the Payments Gateway report to?
FACTS FROM THE GRAPH:
- Payments Gateway is owned by Payments Team
- Payments Gateway is owned by Payments Team; Payments Team is led by Priya Raman; Priya Raman reports to Dana Okafor
- Payments Gateway is owned by Payments Team; Payments Team is led by Priya Raman
- Payments Gateway is owned by Payments Team; INC-2060 affected Payments Gateway
- Payments Gateway is owned by Payments Team; Checkout Service depends on Payments Gateway
- Payments Gateway is owned by Payments Team; Payments Gateway is hosted on Arcadia Cloud
SOURCE PASSAGES:
[svc-payments-gateway] Payments Gateway. The Payments Gateway talks to the card networks on behalf of every other service. The Payments Gateway depends on the Ledger Service to record each authorised charge. The Payments Gateway is owned by the Payments Team. It is hosted on Arcadia Cloud.
[team-payments] Payments Team. The Payments Team looks after card acceptance and refunds. The Payments Team is led by Priya Raman. The Payments Team is based in Pune. The team holds a weekly review on Thursdays.
[person-priya] Priya Raman. Priya Raman joined the company in 2021 as a payments engineer. Priya Raman reports to Dana Okafor. She is the named contact for card network certificate renewals.
[inc-2060] INC-2060. INC-2060 was a 12-minute slowdown on card payments. INC-2060 affected the Card Gateway. The Card Gateway is the on-call channel's name for the Payments Gateway. A retry storm after a network blip was the trigger. The write-up recommends capping retries.
[svc-checkout] Checkout Service. The Checkout Service takes a customer's cart and turns it into an order. It depends on the Payments Gateway to take card payments and on the Inventory Service to reserve stock. The Checkout Service is owned by the Storefront Team. Its latency dashboard lives in the shared Grafana folder.
The closing line compares size: 1,922 characters of prompt, against 1,820 characters for the eight chunks a top-8 vector search would have contributed. Roughly the same context, built from a different selection.
Should you build this?
Only if your questions are chains, and your documents state the links. GraphRAG earned its place here on questions of two and three hops about named things. On the one-hop question every method was equal, and plain vector RAG with a reranker or hybrid search is far cheaper to run. Before building a graph, check three things.
1. Is the relation in the text? Extraction only finds relations the documents state. Prose that implies a relationship without saying it gives a sparse graph and nothing to walk.
2. Can you afford one model call per chunk, and per edit? Indexing cost scales with the corpus (35 calls here; 35 × thousands in real life), and re-indexing a changed chunk means re-extracting it. This project’s graph writes only ever add edges (MERGE): an edge whose sentence was deleted from a document stays in the graph, and no code here removes it. Tracking edges per chunk and deleting the ones a chunk no longer supports is real work, and the ingestion post‘s content-hash approach is the place to start.
3. Who owns entity resolution? Step 2 above is where graphs rot: aliases, renames, two teams with the same name. If nobody owns the alias table, the graph decays silently into disconnected islands.
What this post did not test, stated plainly: extraction by a real language model on real prose (the stand-in reads eight sentence patterns, so real-world extraction recall and precision are unmeasured, and will be worse); answer quality (no model wrote an answer); Microsoft-style community summaries and global questions; anything above 35 documents, including latency; and text-to-Cypher.
Run it yourself
Everything is in the graphrag-neo4j directory of the companion repository. With Docker running and JDK 25, ./scripts/run-all.sh regenerates every transcript quoted here (a few minutes: the Neo4j container takes about a minute to start, and the sweep test runs hundreds of retrievals); NEO4J_IMAGE=neo4j:5-community ./scripts/run-all.sh runs it against 5.26. To try a real model, build a Pipeline with any Spring AI ChatModel in place of the stand-in — the constructor takes one — and expect the extraction failures from the earlier section to appear.
No Comments yet!