How to Detect Hallucinations in RAG: a Per-Sentence NLI Gate in SQL

Objective

Your RAG pipeline retrieves passages, an LLM writes an answer, and the answer ships. Some sentences are backed by the passages, some contradict them, and some came from nowhere. The usual check is a second LLM acting as judge, which costs another API call per answer. It can also share the generator's blind spots.

This recipe checks the answer sentence by sentence with a different kind of model: a small natural-language-inference (NLI) cross-encoder that runs inside the database. One SQL statement:

1. pairs each answer sentence with the retrieved passages about the same thing (EMBED + COSINE_SIMILARITY);
2. checks the sentence against those passages with NLI_VERIFY;
3. labels it SUPPORTED, CONTRADICTED, DISPUTED or UNPROVEN, with the passage that decided it.

A second query rolls that up into a faithfulness score and a decision for the whole answer: SHIP, REVIEW or BLOCK. There's no LLM judge and no external call, and the passages never leave the server. RAG Answer Gate gates a single-claim answer. This recipe handles multi-sentence answers and reports which sentence failed.

Before you start: NLI is off by default

Until a model is installed and enabled, every NLI call fails with Inference error: NLI is disabled; install a model and enable NLI. Enable it once:

synapcores --config /etc/synapcores/gateway.toml pull nli-default
synapcores --config /etc/synapcores/gateway.toml models enable-nli nli-default
# then restart the engine

On Docker, set AIDB_NLI_ENABLED=1 and AIDB_NLI_MODEL=nli-default at startup. GET /v1/ai/nli/models should report ready.

Step 1: The retrieved passages

These are the passages your retriever returned for "What is the refund policy for annual plans?". One of them is off-topic, which is typical of real retrieval:

CREATE TABLE IF NOT EXISTS recipe_halgate_passages (
    id INT PRIMARY KEY,
    doc TEXT,
    body TEXT,
    embedding VECTOR(384)
);

INSERT INTO recipe_halgate_passages (id, doc, body, embedding) VALUES
  (1, 'billing-policy#refunds', 'Annual plans can be refunded in full within 30 days of purchase.',
      EMBED('Annual plans can be refunded in full within 30 days of purchase.')),
  (2, 'billing-policy#refunds', 'After 30 days, annual plans are refunded pro rata for the unused full months.',
      EMBED('After 30 days, annual plans are refunded pro rata for the unused full months.')),
  (3, 'billing-policy#refunds', 'Monthly plans are not refundable.',
      EMBED('Monthly plans are not refundable.')),
  (4, 'billing-policy#payouts', 'Refunds are issued to the original payment method within 5 to 10 business days.',
      EMBED('Refunds are issued to the original payment method within 5 to 10 business days.')),
  (5, 'office-faq',             'The support office is closed on public holidays.',
      EMBED('The support office is closed on public holidays.'));

Step 2: The answers, one row per sentence

Your app splits each generated answer into sentences, which any sentence splitter can do. Answer 1 is a typical LLM draft. Answer 2 is a careful one.

CREATE TABLE IF NOT EXISTS recipe_halgate_sentences (
    answer_id INT,
    n INT,
    sentence TEXT,
    embedding VECTOR(384),
    PRIMARY KEY (answer_id, n)
);

INSERT INTO recipe_halgate_sentences (answer_id, n, sentence, embedding) VALUES
  (1, 1, 'You can get a full refund on an annual plan within 30 days of buying it.',
         EMBED('You can get a full refund on an annual plan within 30 days of buying it.')),
  (1, 2, 'After that, annual plans cannot be refunded at all.',
         EMBED('After that, annual plans cannot be refunded at all.')),
  (1, 3, 'The refund goes back to your original payment method.',
         EMBED('The refund goes back to your original payment method.')),
  (1, 4, 'Refunds arrive within 24 hours.',
         EMBED('Refunds arrive within 24 hours.')),
  (1, 5, 'You can also request a refund by phone.',
         EMBED('You can also request a refund by phone.')),
  (1, 6, 'Enterprise customers get a dedicated account manager.',
         EMBED('Enterprise customers get a dedicated account manager.')),
  (2, 1, 'Monthly plans cannot be refunded.',
         EMBED('Monthly plans cannot be refunded.')),
  (2, 2, 'The money is returned to the original payment method.',
         EMBED('The money is returned to the original payment method.'));

In answer 1, sentence 2 reverses the policy, sentence 4 invents a timeline, and sentences 5 and 6 have no source at all.

Step 3: Look at the scores before you set thresholds

Score each sentence against each passage on the same topic, using a similarity floor of 0.45:

SELECT s.answer_id, s.n, p.id AS passage,
       ROUND(COSINE_SIMILARITY(s.embedding, p.embedding), 3) AS similarity,
       ROUND(CAST(JSON_EXTRACT(NLI_CLASSIFY('nli-default', p.body, s.sentence),
             '$.scores.entailment') AS DOUBLE), 3) AS entailment,
       ROUND(CAST(JSON_EXTRACT(NLI_CLASSIFY('nli-default', p.body, s.sentence),
             '$.scores.contradiction') AS DOUBLE), 3) AS contradiction
FROM recipe_halgate_sentences s
CROSS JOIN recipe_halgate_passages p
WHERE s.answer_id = 1
  AND COSINE_SIMILARITY(s.embedding, p.embedding) > 0.45
ORDER BY s.n, similarity DESC;

Observed on 2.3.1 with nli-default:

Four things to notice:

• The off-topic passage 5 never clears the floor: its best similarity to any sentence is 0.235. Without the floor, the model can score an unrelated passage as a "contradiction". See recipe 029 for that trap.
• Sentence 6 clears the floor for no passage. There's no evidence at all.
• Sentences 1 and 4 have evidence on both sides: one passage entails them and another contradicts them. Step 6 explains why.
• Scores cluster near 0 and 1. The 0.6 thresholds used below aren't sensitive here, but check that on your own data.

Step 4: One query, a verdict per sentence

NLI_ANCHORS collects each sentence's on-topic passages, and NLI_VERIFY returns TRUTH, LIE or UNVERIFIED_PLAUSIBLE. NLI_VERIFY lets contradiction override entailment, so the query also takes the best entailment in the same group. A sentence with both is labelled DISPUTED rather than quietly called a lie:

CREATE TABLE IF NOT EXISTS recipe_halgate_results (
    answer_id INT,
    n INT,
    sentence TEXT,
    status TEXT,
    evidence TEXT,
    PRIMARY KEY (answer_id, n)
);

INSERT INTO recipe_halgate_results (answer_id, n, sentence, status, evidence)
SELECT s.answer_id, s.n, s.sentence,
       CASE
         WHEN v.verdict IS NULL THEN 'UNPROVEN'
         WHEN v.verdict = 'TRUTH' THEN 'SUPPORTED'
         WHEN v.verdict = 'LIE' AND v.best_entailment > 0.6 THEN 'DISPUTED'
         WHEN v.verdict = 'LIE' THEN 'CONTRADICTED'
         ELSE 'UNPROVEN'
       END,
       COALESCE(v.violated, v.matched)
FROM recipe_halgate_sentences s
LEFT JOIN (
  SELECT x.answer_id, x.n,
         JSON_UNQUOTE(JSON_EXTRACT(NLI_VERIFY('nli-default', x.sentence, NLI_ANCHORS(p.body)),
              '$.verdict')) AS verdict,
         JSON_UNQUOTE(JSON_EXTRACT(NLI_VERIFY('nli-default', x.sentence, NLI_ANCHORS(p.body)),
              '$.violated_anchor')) AS violated,
         JSON_UNQUOTE(JSON_EXTRACT(NLI_VERIFY('nli-default', x.sentence, NLI_ANCHORS(p.body)),
              '$.matched_anchor')) AS matched,
         MAX(CAST(JSON_EXTRACT(NLI_CLASSIFY('nli-default', p.body, x.sentence),
              '$.scores.entailment') AS DOUBLE)) AS best_entailment
  FROM recipe_halgate_sentences x
  CROSS JOIN recipe_halgate_passages p
  WHERE COSINE_SIMILARITY(x.embedding, p.embedding) > 0.45
  GROUP BY x.answer_id, x.n, x.sentence
) v ON v.answer_id = s.answer_id AND v.n = s.n;

SELECT answer_id, n, status, sentence, evidence
FROM recipe_halgate_results
ORDER BY answer_id, n;

Observed:

The evidence column is the passage that decided the verdict: the one contradicted for CONTRADICTED and DISPUTED, the one that supports the sentence for SUPPORTED. Sentence 5 had on-topic passages, but none supported or contradicted it. Sentence 6 had no on-topic passage at all. Both are UNPROVEN: the retrieved evidence doesn't back them.

Step 5: Faithfulness score and the gate decision

SELECT answer_id,
       COUNT(*) AS sentences,
       SUM(CASE WHEN status = 'SUPPORTED' THEN 1 ELSE 0 END) AS supported,
       ROUND(SUM(CASE WHEN status = 'SUPPORTED' THEN 1 ELSE 0 END) * 1.0 / COUNT(*), 2) AS faithfulness,
       CASE
         WHEN SUM(CASE WHEN status = 'CONTRADICTED' THEN 1 ELSE 0 END) > 0 THEN 'BLOCK'
         WHEN SUM(CASE WHEN status = 'SUPPORTED' THEN 1 ELSE 0 END) = COUNT(*) THEN 'SHIP'
         ELSE 'REVIEW'
       END AS gate
FROM recipe_halgate_results
GROUP BY answer_id
ORDER BY answer_id;

Observed:

Answer 1 is blocked because it reverses the refund policy. If sentence 2 were fixed, it would still go to REVIEW for the unproven and disputed sentences. Answer 2 ships. Faithfulness is the share of sentences the passages support. It's an evidence score you can log per answer and trend over time, not a measure of truth.

Step 6: What DISPUTED means, in this data

Both DISPUTED sentences are worth a close look, because they show where the NLI model is unreliable:

• Sentence 1 is a false alarm caused by scope. Passage 1 entails it at 0.998. Passage 2 ("after 30 days … pro rata") scores 0.995 contradiction against "within 30 days … full refund", but the two don't conflict. The model misses that one rule covers the first 30 days and the other covers the time after. Rewording passage 2 as "Annual plans cancelled after the first 30 days receive a pro-rata refund…" didn't help, scoring 0.995 again.
• Sentence 4 is a real error flagged for the wrong reason. Passage 4 says "within 5 to 10 business days", and the model scores "within 24 hours" as entailed (0.930). It doesn't compare numbers. The sentence ended up DISPUTED only because passage 1 happened to contradict it.

DISPUTED doesn't mean the sentence is wrong. It means the evidence disagrees, so the gate sends it to REVIEW, never SHIP. If you treat every LIE as CONTRADICTED and block on it, as raw NLI_VERIFY would, sentence 1 would block correct answers like "Annual plans are fully refundable within 30 days of purchase."

FAQ: NLI model vs LLM judge

Use NLI as the fast gate on every answer, and send the sentences labelled DISPUTED or UNPROVEN to a human, or to an LLM judge if you have one.

Cleanup (Optional)

DROP TABLE IF EXISTS recipe_halgate_passages;
DROP TABLE IF EXISTS recipe_halgate_sentences;
DROP TABLE IF EXISTS recipe_halgate_results;

Honest boundaries

• Support, not truth. SUPPORTED means the retrieved passages entail the sentence. If the passages are wrong or stale, the check passes the error.
• Retrieval decides what can be checked. A true sentence whose passage wasn't retrieved comes back UNPROVEN. Check against the same passages the generator saw, not a fresh search.
• Numbers and dates need exact checks. Step 6 showed "24 hours" scored as entailed by "5 to 10 business days" (0.930). Extract quantities and compare them as typed values.
• Choose thresholds from your data. The 0.45 similarity floor sits between the off-topic passage (at most 0.235 against any sentence) and the weakest passage this data needs (0.561, the one that catches sentence 2). Re-run Step 3 on a labelled sample of your own answers before relying on the gate.
• On 2.3.1, filter similarity in WHERE over a CROSS JOIN. The same condition in JOIN … ON returned no rows on that build.
• English model. NLI is off by default, so enable it as described above.

Key Concepts Learned

• Check each sentence against its own on-topic passages. A whole-answer verdict can't tell you which sentence to remove.
• NLI_VERIFY lets contradiction override entailment. That suits a strict gate, but adding MAX(entailment) in the same GROUP BY lets you tell "contradicted" apart from "the evidence disagrees".
• Faithfulness (supported sentences ÷ all sentences) and the SHIP / REVIEW / BLOCK decision come from one more GROUP BY. The passages, embeddings, verifier and gate all run in the same engine.