How to Check If an AI Answer Is Actually Backed by Your Documents

The problem: your AI assistant answers a customer from your own documents. The answer sounds right and cites a real policy. But is every sentence actually in that policy? You need a check you can run on every answer, before the customer sees it.

We build SynapCores. All the numbers below come from our own recipes, with the engine version and models stated for each. The Python section shows how to reproduce the lesson without our engine.

Why a retrieved document doesn't prove the answer

Retrieval-augmented generation (RAG) gives the model the right source to copy from. It doesn't check that the model stayed with that source. The model can reverse a rule, change a number or blend two passages, and the retrieved passage still looks like a match. Most "grounded" checks then compare the answer with that passage by similarity, which is the step that fails below.

The same mistake shows up in other places:

• RAG groundedness checks that say "the answer is grounded if the top passage scores above X".
• Agent memory that decides a new fact is "the same as" or "replaces" an old one because they are similar.
• Duplicate and conflict detection across documents, tickets or witness statements.

In all three, a similarity score is quietly used as if it were evidence.

What NLI is, in one paragraph

An NLI model reads two pieces of text, a premise (a passage from your documents) and a claim (one sentence of the AI answer). It returns three scores that add up to 1: entailment (the passage supports the claim), contradiction (the passage says the opposite) and neutral (the passage doesn't settle it). It is a small model, it runs on a CPU, and it gives the same scores every time. It knows nothing about the world. It only judges the passage you hand it, so it answers "is this backed by my documents?", not "is this true?".

What each signal actually measures

                 "Is it on topic?"      "Does the source support it?"    "Is it true?"
                 ─────────────────      ─────────────────────────────    ─────────────
 tool            vector similarity      NLI (entailment/contradiction)   source quality
                                                                         + exact checks
                                                                         + people
 good at         finding candidates     judging a claim vs a passage     —
 blind to        negation, numbers,     topic (unrelated pairs),          —
                 who/when/scope         numbers, scope

The data

Everything below comes from three of our recipes, run with the default 384-dim embedding model and the nli-default NLI model (DeBERTa-v3-small, English):

• Recipe 037 (RAG hallucination gate, observed on 2.3.1, re-run on 2.3.2-ce): five retrieved refund-policy passages and a six-sentence LLM-style answer about annual-plan refunds.
• Recipe 036 (agent memory contradiction check, observed on 2.3.1, re-run on 2.3.2-ce): four stored memories about a customer and six incoming facts.
• Recipe 034 (statement contradiction mining, observed on v2.1.2-ce, re-run on 2.3.2-ce): four witness statements.

Every score quoted below was re-run on 2.3.2-ce (build 563) and came out the same as on the earlier version, except the reworded-passage score in Failure 3.

The passages from recipe 037:

Failure 1: similarity scores a wrong answer as close

Here are four sentences of the generated answer, each with the passage it is most similar to (recipe 037, Step 3, 2.3.1):

A "grounded if similarity > 0.7" rule passes all four, including both wrong sentences. Sentence 4 even outscores sentence 3, which is correct.

Look closer at sentence 2. The passage that contradicts it (passage 2) is its least similar on-topic passage, at 0.561. Passages 3 and 1 score higher (0.736 and 0.727), because they share the words and the topic. Similarity ranks the evidence that would catch the error below evidence that is merely related.

Why: an embedding places texts by topic and vocabulary. "Can be refunded" and "cannot be refunded" are about the same thing, so they land close together. Negation, numbers and scope barely move the vector.

Failure 2: the NLI model calls unrelated sentences contradictions

So use a model that judges support instead. An NLI model takes a premise and a claim and returns scores for entailment, contradiction and neutral. It catches sentence 2 straight away: contradiction 0.999 against passage 2 (recipe 037, Step 3).

But run it on pairs that were never comparable and it still answers confidently. From recipe 036, Step 3 (2.3.1), the incoming fact "The customer is learning to play the cello." against four stored memories:

The cello "contradicts" everything. Of the nine incoming/memory pairs scored above 0.6 contradiction in that recipe, only three were real conflicts. Recipe 034 shows the same thing with witness statements: "The north parking garage was closed for repairs all week." scored ≈ 0.999 contradiction against two unrelated statements, above both real contradictions (≈ 0.995 and ≈ 0.990) (observed on v2.1.2-ce).

Why: a pairwise NLI model is trained on sentence pairs that were meant to be compared. It was never asked "are these about the same thing?", so it has no way to say "unrelated".

The fix is the thing that failed in Failure 1. Similarity is a poor judge of truth but a good judge of relevance. In recipe 036, real conflicts sat at similarity 0.50–0.74 and false ones at 0.41 or below, so a floor of 0.45 removed every false pair. With the floor in place, all six incoming facts were sorted correctly: three conflicts, one already known, two new (recipe 036, Step 5).

Failure 3: NLI misreads scope

Sentence 1 of the answer ("full refund … within 30 days") is correct. Passage 1 entails it at 0.998. But passage 2 ("after 30 days … pro rata") scores 0.995 contradiction against it (recipe 037, Step 3). The two rules don't conflict: one covers the first 30 days, the other the time after. The model doesn't see that. Rewording passage 2 as "Annual plans cancelled after the first 30 days receive a pro-rata refund." still scored 0.994 (0.990 with "for the unused full months" added; recipe 037, Step 6, re-run on 2.3.2-ce).

If you block every answer with any contradiction, you block correct answers. Recipe 037 handles this by labelling a sentence with both strong entailment and strong contradiction as DISPUTED and sending it to review instead of blocking it.

Failure 4: NLI doesn't compare numbers

Back to sentence 4, "Refunds arrive within 24 hours." The NLI model scores it as entailed by "within 5 to 10 business days" at 0.930 (recipe 037, Step 3). Similarity missed it, and NLI missed it too. It only ended up flagged because passage 1 happened to contradict it (0.977), for an unrelated reason.

Neither tool compares quantities. Extract numbers and dates and compare them as typed values.

What holds up: a layered check

The thresholds are a trade-off. In our internal tests on agent memory, stricter settings (a higher similarity floor, a higher contradiction threshold, and replacing only the single closest memory) avoided wrongly overwriting good facts, at the cost of missing some real updates. Tune them on your own data, and send the cases you're unsure about to review rather than overwriting.

When this doesn't matter

Similarity on its own is fine when a person reads the results: search boxes, "related articles", recommendations, finding near-duplicate tickets for an agent to merge. It's also fine when the claims are structured: if both sides are rows with typed columns, compare the columns and skip NLI entirely. And if you already run an LLM judge with human review and the cost works at your volume, you may not need an NLI layer.

A fair comparison of checks

The LLM-judge row is a general expectation, not a measurement. We haven't benchmarked LLM judges on this data.

Reproduce it without our database

You can see the same effect with open-source models in Python. The embedding model below is the common 384-dim MiniLM; the NLI model is the cross-encoder our engine bundles as nli-default.

# pip install "sentence-transformers==5.1.2"
from sentence_transformers import SentenceTransformer, CrossEncoder, util

emb = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
nli = CrossEncoder("cross-encoder/nli-deberta-v3-small")
LABELS = ["contradiction", "entailment", "neutral"]

pairs = [
    ("Refunds are issued to the original payment method within 5 to 10 business days.",
     "Refunds arrive within 24 hours."),
    ("After 30 days, annual plans are refunded pro rata for the unused full months.",
     "After that, annual plans cannot be refunded at all."),
    ("The customer's office is in Denver.",
     "The customer is learning to play the cello."),
]

for premise, claim in pairs:
    sim = util.cos_sim(emb.encode(premise), emb.encode(claim)).item()
    probs = nli.predict([(premise, claim)], apply_softmax=True)[0]
    scores = {l: round(float(p), 3) for l, p in zip(LABELS, probs)}
    print(f"{sim:.3f}  {scores}  | {claim}")

Output (sentence-transformers 5.1.2, torch 2.8.0, transformers 4.57.6, Python 3.9, CPU). The model's label order is contradiction / entailment / neutral (id2label in its config), and the scores match the engine's to three decimals:

0.779  {'contradiction': 0.059, 'entailment': 0.93, 'neutral': 0.012}  | Refunds arrive within 24 hours.
0.561  {'contradiction': 0.999, 'entailment': 0.0, 'neutral': 0.001}  | After that, annual plans cannot be refunded at all.
0.235  {'contradiction': 0.997, 'entailment': 0.0, 'neutral': 0.003}  | The customer is learning to play the cello.

The same check in one SQL statement (SynapCores)

On SynapCores the embeddings, the NLI model and the data are in one engine, so the layered check is one query over a whole batch. This is the gated memory check from recipe 036, Step 5: each incoming fact is compared only with memories above the 0.45 similarity floor, NLI_VERIFY judges it against all of them at once, and a LEFT JOIN keeps facts with no related memory as NEW. Table setup is in recipe 036, Steps 1–2.

SELECT i.id, i.source,
       CASE g.verdict
         WHEN 'LIE'   THEN 'CONFLICT'
         WHEN 'TRUTH' THEN 'ALREADY_KNOWN'
         ELSE 'NEW'
       END AS status,
       g.conflicts_with
FROM recipe_memcheck_incoming i
LEFT JOIN (
  SELECT c.id AS cand_id,
         JSON_UNQUOTE(JSON_EXTRACT(NLI_VERIFY('nli-default', c.fact, NLI_ANCHORS(m.fact)),
              '$.verdict')) AS verdict,
         JSON_UNQUOTE(JSON_EXTRACT(NLI_VERIFY('nli-default', c.fact, NLI_ANCHORS(m.fact)),
              '$.violated_anchor')) AS conflicts_with
  FROM recipe_memcheck_incoming c
  CROSS JOIN recipe_memcheck_memories m
  WHERE COSINE_SIMILARITY(c.embedding, m.embedding) > 0.45
  GROUP BY c.id, c.fact
) g ON g.cand_id = i.id
ORDER BY i.id;

Output on 2.3.2-ce (build 563), the same as recipe 036 recorded on 2.3.1:

Without the WHERE COSINE_SIMILARITY(...) > 0.45 line, the same query rejected five of six facts, including the cello and the webinar (recipe 036, Step 4). The numbers check (Failure 4) is not in this query; add it as a typed comparison.

Where to go next

• Before this: How to Search by Meaning Instead of Keywords (what similarity scores are)
• Before this: How to Check If Two Sentences Mean the Same Thing (or Contradict Each Other)Coming soon (NLI and embeddings side by side)
• Next: How to Build a Support Chatbot That Only Answers From Your Help Articles (With Sources)Coming soon
• Next: How to Make a Chatbot Remember Previous Conversations (the same contradiction check, applied to memory)
• All topics: Which Technique Solves My Problem?

Run it yourself: recipe 037, How to Detect Hallucinations in RAG: a Per-Sentence NLI Gate in SQL, builds layers 2 and 3 end to end on the refund-policy data above, and turns the result into a ship / review / block decision. Recipe 038 checks that each cited passage supports the sentence that cites it.