GraphRAG vs Neo4j: a knowledge base that answers multi-hop questions

This is the hub of a 5-part GraphRAG cluster. It teaches the core pattern; the spokes apply it to real problems — fraud-ring detection, supply-chain impact analysis, customer-360 Q&A, and product recommendations. For the architecture argument in depth, read the pillar: GraphRAG vs Neo4j.

Objective

Vector-only RAG retrieves passages by similarity but throws away the relationships between them. Ask "which runbooks apply to a database outage on a service my team owns?" and pure vector search finds documents that sound relevant — but it can't enforce "my team owns it" or "it's about the database tier," because those are edges in a graph, not words in a passage.

GraphRAG fixes this by combining semantic recall with graph structure. The textbook way to build it is a four-system stack: Neo4j for the graph, a vector database (Pinecone / Weaviate / pgvector) for embeddings, an orchestration layer (LangChain / LlamaIndex) to glue them, and an LLM API for ranking and synthesis. Four services, three network hops per query, and a sync job to keep the graph and the vectors from drifting apart.

SynapCores collapses that into one query on one engine: nodes carry their own embedding property, [:SIMILAR_TO > 0.85] is a semantic hop inside Cypher (HNSW under the hood), and llm_score(prompt, node) grades relevance inline. This recipe builds a small company knowledge base and answers a multi-hop question with a single Cypher statement — then shows you exactly what the Neo4j stack version would take.

Step 1: Build the knowledge graph

Documents (runbooks/policies) carry an embedding and link to the systems they cover and the teams that own them. Embedding values here are 5-dimensional demo vectors so the recipe is fast and deterministic; in production you generate them with EMBED('the document text') (384-dim by default).

// Docs — each with a semantic embedding of its content
MERGE (d1:KbDoc {id: "DOC-101", title: "Postgres failover runbook",
                 text: "Steps to promote a replica when the primary database is unresponsive",
                 embedding: [0.82, 0.10, 0.05, -0.06, 0.20]})
MERGE (d2:KbDoc {id: "DOC-102", title: "Database connection pool exhaustion",
                 text: "Diagnose and resolve exhausted connection pools on the database tier",
                 embedding: [0.79, 0.12, 0.03, -0.04, 0.22]})
MERGE (d3:KbDoc {id: "DOC-201", title: "CDN cache purge guide",
                 text: "How to purge and warm the CDN edge cache after a bad deploy",
                 embedding: [-0.31, 0.60, 0.18, 0.05, 0.11]})
MERGE (d4:KbDoc {id: "DOC-202", title: "TLS certificate rotation",
                 text: "Rotate expiring TLS certificates on the edge load balancers",
                 embedding: [-0.28, 0.57, 0.20, 0.07, 0.09]})
MERGE (d5:KbDoc {id: "DOC-301", title: "Incident comms policy",
                 text: "Who to notify and when during a customer-facing incident",
                 embedding: [0.06, -0.42, 0.65, 0.09, -0.17]})

// Systems the docs cover
MERGE (s1:KbSystem {name: "Payments DB", tier: "database"})
MERGE (s2:KbSystem {name: "Edge CDN",    tier: "edge"})

// Teams that own systems
MERGE (t1:KbTeam {name: "Core Data", oncall: "core-data-oncall"})
MERGE (t2:KbTeam {name: "Edge Platform", oncall: "edge-oncall"})

// Structure: docs COVER systems, systems are OWNED_BY teams
MERGE (d1)-[:COVERS]->(s1)
MERGE (d2)-[:COVERS]->(s1)
MERGE (d3)-[:COVERS]->(s2)
MERGE (d4)-[:COVERS]->(s2)
MERGE (s1)-[:OWNED_BY]->(t1)
MERGE (s2)-[:OWNED_BY]->(t2);

Step 2: The vector half — semantic recall alone

[:SIMILAR_TO > 0.85] walks from a seed document to the documents whose embeddings are near it. This is the "vector database" part — except it's a Cypher edge, not a separate service.

// "The on-call engineer is reading the Postgres failover runbook.
//  What other docs are semantically related?"
MATCH (seed:KbDoc {id: "DOC-101"})-[:SIMILAR_TO > 0.85]->(related:KbDoc)
RETURN related.id AS id, related.title AS title;

Expected: DOC-102 (connection-pool exhaustion) — the other database-tier doc. The CDN and comms docs are semantically far, so they don't come back.

Step 3: GraphRAG — semantic recall + graph structure + LLM ranking, one query

Now the payoff. Start from the seed doc, hop to its semantic neighbors, walk the structural edges to the owning team, and let llm_score rank how directly each doc addresses a database-outage — all in a single statement.

// "For a database-tier incident, find the most relevant runbooks and who owns them."
MATCH (seed:KbDoc {id: "DOC-101"})-[:SIMILAR_TO > 0.8]->(doc:KbDoc)
MATCH (doc)-[:COVERS]->(s:KbSystem)-[:OWNED_BY]->(team:KbTeam)
WHERE s.tier = "database"
WITH doc, team,
     llm_score("How directly does this runbook help resolve a database outage? 0=unrelated, 1=exactly the fix", doc) AS relevance
WHERE relevance > 0.4
RETURN doc.title AS runbook,
       team.name AS owning_team,
       team.oncall AS page,
       relevance
ORDER BY relevance DESC;

Expected: the related database runbook (DOC-102, connection-pool exhaustion) surfaces with its owning team + on-call handle and a high LLM relevance score (~0.9). The seed doc is excluded from its own neighbor set, so you get related runbooks, not the one you started from. Vector-only RAG could find the runbook, but it could not attach "owned by Core Data, page core-data-oncall" — that lives in the graph.

What this takes on a Neo4j + vector-DB stack (for reference — not executed)

The same capability on the classic stack spans four systems. This block is a reference comparison, not runnable SynapCores code:

# 1) Neo4j holds the graph (Cypher) — but Neo4j has no native embedding hop,
#    so SIMILAR_TO does not exist. You store doc IDs and structure only:
#    (Neo4j Cypher)
MATCH (doc:KbDoc)-[:COVERS]->(sys:System {tier:'database'})-[:OWNED_BY]->(team:Team)
RETURN doc.id, team.name

# 2) A separate vector DB (Pinecone/Weaviate/pgvector) holds the embeddings.
#    You query it out-of-band for semantic neighbors:
index.query(vector=embed("postgres failover"), top_k=10)   # -> [doc ids]

# 3) Application/orchestration code (LangChain) intersects the two result sets,
#    because neither system can see the other's data:
ids = set(vector_ids) & set(graph_ids)

# 4) A separate LLM API call re-ranks each surviving doc:
for d in ids: score = openai.rank(prompt, d)   # N more network calls

# Then keep Neo4j and the vector DB in sync forever with a CDC/ETL job.

Four systems, three-to-N network round trips per question, and a synchronization job. In SynapCores it was one Cypher query against one store — the graph and the vectors are the same data. That is the whole argument of GraphRAG vs Neo4j.

Step 4: One more — combine structure and semantics freely

// "Docs semantically near the CDN cache guide, but only ones owned by Edge Platform."
MATCH (seed:KbDoc {id: "DOC-201"})-[:SIMILAR_TO > 0.8]->(doc:KbDoc)
MATCH (doc)-[:COVERS]->(:KbSystem)-[:OWNED_BY]->(team:KbTeam {name: "Edge Platform"})
RETURN doc.title AS runbook, team.name AS team;

Cleanup (Optional)

MATCH (n:KbDoc) DETACH DELETE n;
MATCH (n:KbSystem) DETACH DELETE n;
MATCH (n:KbTeam) DETACH DELETE n;

Use it from your app or agent

• In production: replace the demo vectors with EMBED('doc text') when you MERGE nodes (384-dim), and add CREATE VECTOR INDEX for large catalogs. The Cypher queries above don't change.
• REST/SDK: every query here is a POST /v1/graph/match with {"sql": "<cypher>"}, or client.graph.cypher(...) in the Python SDK — one call, one auth token.
• Why one engine: the graph edges and the embeddings are the same rows in the same store, so there is no vector-DB sync job and no app-side set-intersection. Add an LLM step with llm_score without leaving the query.

Key Concepts Learned

• GraphRAG = semantic recall + graph structure + LLM ranking. Vector-only RAG has the first, misses the second and third.
• [:SIMILAR_TO > t] is an inline vector hop — the vector DB collapses into a Cypher edge, with a tunable threshold for precision/recall.
• llm_score(prompt, node) ranks inside the query — no app-side LLM loop.
• The Neo4j stack needs four systems and a sync job for the same result; here the graph and vectors are one dataset.

Next in this cluster: Fraud-ring detection · Supply-chain impact · Customer-360 Q&A · Product recommendations · Pillar: GraphRAG vs Neo4j