The most common architectural mistake I see in AI applications today is treating vector search as a complete retrieval system. It's not. Vector search is a complete similarity system, which is a different thing. Similarity is a useful signal. It is not a sufficient signal for most non-trivial retrieval problems, and the gap between "we found similar things" and "we found the right things" is exactly where vector-only RAG systems disappoint their users.

That gap closes when you add graph relationships to the retrieval path. Not as a replacement for the vector index — as a complement. The vector index is what gets you to the neighborhood; the graph is what tells you which streets in the neighborhood are actually connected to the one you came from.

This post is the case for adding graph relationships to your vector search, the four signals that say you need to, what "adding a graph" actually looks like operationally, and how to do it without bolting on a second database. The examples are SynapCores SQLv2 because that's the dialect where vectors and graphs live in one engine, but the argument applies whether you adopt our engine or stack two of them yourself.

The two questions retrieval can answer

Vector search answers exactly one question: what items in my corpus are most similar to this query item?

Graph traversal answers a different question: what items in my corpus are connected to this seed item, and along what relationships?

Both questions are useful. They are not the same question. A retrieval system that can only answer one of them is incomplete by design. The simple statement of the post is: most production AI applications need both, and people keep building systems with only one.

For the more concrete version of why vector-only retrieval fails on relational questions, see Why Vector Search Alone Fails Complex Reasoning.

Four signals you've outgrown vector-only

I keep a running list from real production incident review. If your application hits any of these, you've reached the point where vector-only stops paying back.

1. The "what about X" question

A support copilot finds a relevant chunk: "Customer 42 reported MRI integration timeouts in March." The user follows up: "What about other accounts on the same plan?" The vector search has no way to express the same plan as a constraint. It can search for "Pro plan customers" semantically, but it can't follow the edge from Customer 42 to Plan to other Customers on the same Plan.

A graph traversal expresses this directly:

MATCH (c:Customer {id: 42})-[:ON_PLAN]->(p:Plan)<-[:ON_PLAN]-(other:Customer)
RETURN other

The vector index can't produce this answer because the relationship :ON_PLAN isn't in the embedding space. It's in the schema.

2. The "through whom" question

"How is this lead connected to our existing customers?" "Through what chain of approvals did this PR merge?" "What's the path of evidence from this transaction to the flagged ring?"

These all require the path itself to be in the answer. Vector search returns a set; the user wants a chain. Without graph traversal, the application has to reconstruct chains by repeated vector lookups — which is slow, lossy, and a lot of code.

3. The "at what time" question

A user asks: "Did the price of this product change in the last 30 days, and if so, what was it before?"

Vector search embeds the current text. It has no temporal model — versions and validity windows live in your schema, not in the embedding. Graph edges with valid_from / valid_to properties solve this cleanly:

MATCH (p:Product {id: 'abc'})-[r:HAS_PRICE]->(price:Price)
WHERE r.valid_from < datetime('30 days ago')
   OR r.valid_to > datetime('30 days ago')
RETURN price.value, r.valid_from, r.valid_to
ORDER BY r.valid_from DESC

You can simulate this in vector land by embedding multiple versions and filtering by timestamp, but you're outside the vector index doing that — and you've now lost the ranking signal that made vector search useful.

4. The "why did the agent say that" question

Audit. Every agent that ever ships hits this question. Users want to know which information led to a given answer. With vector-only, you can say "we used chunks 4837, 5012, 6219." With graph-aware retrieval, you can say "we walked Account 42 → BELONGS_TO → Org 7 → HAD_INCIDENT → Incident 19 → DESCRIBED_BY → chunk 4837." The path is the explanation.

If your product has any compliance or trust dimension — and most do, eventually — path-of-evidence beats chunk-id-list every time.

What "adding a graph" actually means

This is where the conversation usually stalls because people think "adding a graph" means deploying Neo4j. It doesn't have to. There are three operational shapes:

Shape 1: dedicated graph database

Deploy Neo4j (or Memgraph, JanusGraph, etc.) next to your vector store. Two clusters, two query languages, two operational footprints, a sync job between them.

This works and the tooling is mature. The cost is the seams, as covered in GraphRAG vs Neo4j.

Shape 2: multi-model database with both

ArangoDB, OrientDB, and a few others bake both graph and document/vector capabilities into one engine. Lower operational footprint than two clusters. Smaller ecosystem, smaller community, smaller bench of integrations.

Shape 3: unified AI-native engine

The graph engine and the vector index share storage and a planner. Cypher and SQL operate on the same data; you can express the whole retrieval in one query.

WITH seeds AS (
  SELECT id FROM chunks
  WHERE COSINE_SIMILARITY(embedding, EMBED(:question)) > 0.6
  ORDER BY similarity DESC LIMIT 5
)
SELECT c.body, n.label, e.type, m.label
FROM seeds s
GRAPH MATCH (Chunk {id: s.id})-[e*1..2]-(m:Entity)
LIMIT 20;

This is what SynapCores does. It's not the only engine in this category — the category is growing — but it's the operational shape that has the lowest seam count.

The decision isn't whether to add graph relationships. It's which shape to add them in.

A concrete walkthrough: extending a vector RAG with graph

Let me make this concrete. Say you have a working vector RAG. You're using Pinecone for vectors, OpenAI for the LLM, and a Postgres table for the chunks themselves. The system works for "factual" questions and falls over on relational questions like the four above. You decide to add graph relationships.

Step A: identify the entities

Walk your chunks and extract the entities each chunk mentions. Use the LLM if needed:

def extract_entities(chunk_text):
    prompt = f"Extract entities (Person, Company, Product, etc.) from: {chunk_text}\nReturn JSON: [...]"
    return json.loads(llm.complete(prompt))

You now have, for each chunk, a list of entities. Store them as nodes; store the (chunk → MENTIONS → entity) edges.

Step B: identify the relationships between entities

This is the harder step. What relationships matter in your domain? Customer-PARENT_ORG-Customer? Product-VARIANT_OF-Product? Ticket-RESOLVED_BY-Agent? You have to decide, because the embedding model can't decide for you.

This is the schema-design phase and it's the part most teams underestimate. Spend a week here. Get it wrong and your graph is decoration; get it right and the relational questions start answering themselves.

Step C: write the relationships

For each pair of entities, decide the relationship and write the edge. Sometimes this is automated (your CRM already knows which customer belongs to which org); sometimes it's LLM-assisted; sometimes it's manual curation. Hybrid is usually right.

Step D: extend the retrieval

Add a graph-traversal step after the vector seed. In the dual-store shape, this is a second query to Neo4j. In the unified-engine shape, it's a clause in the same SQL statement.

Step E: evaluate

Build an eval set of 50 questions, half single-chunk (vector-only suffices) and half relational (graph helps). Score before and after. The improvement on the relational half is the value of the graph; the no-regression on the single-chunk half tells you the graph didn't break anything.

What this looks like in practice

I worked through a version of this on a call-routing-agent shape — past-call-transcript retrieval that was vector-only. The system was good at "find calls similar to this one" and bad at "find calls from the same customer about the same product line." Adding entities (Customer, Product, Issue) and edges (FROM_CUSTOMER, ABOUT_PRODUCT, ESCALATED_TO) closed the gap on the latter without touching the former.

The total code change was about 200 lines of Python for the extraction, ~80 lines of SQL for the schema, and a one-line edit to the retrieval query (an added GRAPH MATCH clause). The improvement on relational queries was substantial; the latency cost was about +35ms on average.

The recipe is published in our library — run it on the included sample data and you'll see the same before/after numbers without taking my word for it.

The diagram I draw on whiteboards

Vector retrieval is a circle in embedding space — radius determined by cosine > threshold. Graph traversal is a tree-walk from points inside that circle, hopping along typed edges. The retrieval result is both: the chunks inside the circle plus the nodes and edges reachable from them.

When I draw this for engineers, the moment they "get it" is when I say the vector tells you where to start the walk; the graph tells you where to walk to.

What if your data doesn't have real relationships?

This is the honest pushback. Not every corpus has natural graph structure. A pile of FAQ documents is probably just a pile. A library of standalone news articles, similarly. If you don't have entities worth modeling and relationships worth traversing, adding a graph is overhead with no payoff.

The test is simple: take 50 user questions from your real product, manually answer each one, and note which ones required following a relationship. If less than ~10% involve relationships, you don't need a graph. Save the work for when you do.

If between 10% and 30% involve relationships, it's a judgement call. The graph helps but isn't required.

If more than 30%, your eval set is screaming for a graph. Add one.

Where this fits in the bigger picture

The pattern "vector seed + graph expand" is what most people now call GraphRAG. The cluster anchor is What Is GraphRAG?; the comparison is GraphRAG vs Traditional RAG; the deeper technical primer is Why Vector Search Alone Fails Complex Reasoning.

This post is the bridge — the "we need to talk about adding graph relationships" conversation. If the cluster lands and the bridge is convincing, the next click for the reader is one of the three above.

The 5-minute experiment

If you want to see the difference on your data without committing to an architecture change:

Download SynapCores Community Edition.
Import a CSV or markdown corpus.
Run recipe 015_vector_search — that's vector-only retrieval.
Run recipe 016_graphrag_qna on the same data — that's vector + graph.
Ask both the same 10 relational questions. Compare.

That's the demo. If your data doesn't benefit, you saved yourself a quarter of engineering work. If it does, you have evidence to bring to the architecture review.

The honest version

We don't have customer logos yet. The case for adding graph relationships is structural, not promotional. If you want help on the schema-design step (which is, again, the hard part most teams underestimate), the Agent Memory JumpStart is the 2-4 week founder-led sprint where we wire vector + graph retrieval into your real workflow. Free Design Partner track or fixed-fee Paid Pilot from $5,000. The schema design is the part the JumpStart actually delivers — once that's right, the SQL is straightforward.

Or just run the two recipes back-to-back. If the second one doesn't beat the first on your relational questions, the graph isn't earning its keep — and that's a useful answer too.

Why Vector Search Needs Graph Relationships