Incident Report Integrity: Postmortem Claims vs. the Timeline

Objective

A postmortem is only as trustworthy as its grounding in the timeline. Under deadline, narrative claims drift from the evidence — a root cause asserted that the logs don't show, a reassuring "no customer impact" nobody actually verified. This recipe matches each claim in the report to the timeline entry it concerns and separates the grounded conclusions from the ones that need evidence before sign-off.

Step 1: The recorded timeline

CREATE TABLE IF NOT EXISTS recipe_incident_timeline (
    id INT PRIMARY KEY,
    ts TEXT,
    event TEXT
);

INSERT INTO recipe_incident_timeline (id, ts, event) VALUES
  (1, '02:14', 'The primary database failed over to the replica, triggering the outage.'),
  (2, '02:19', 'Automated alerts fired and the on-call engineer acknowledged.'),
  (3, '02:41', 'The replica was promoted and traffic recovered.'),
  (4, '03:05', 'A backlog of queued writes finished draining.');

Step 2: Report claims, each matched to the timeline entry it concerns

CREATE TABLE IF NOT EXISTS recipe_incident_claims (
    id INT PRIMARY KEY,
    claim TEXT,
    timeline_id INT
);

INSERT INTO recipe_incident_claims (id, claim, timeline_id) VALUES
  (1, 'The failover to the replica triggered the outage.',   1),
  (2, 'The incident was caused by a configuration change.',  1),
  (3, 'Traffic recovered after the replica was promoted.',   3);

!!! note "Match, then verify — don't dump the whole timeline" NLI_VERIFY returns LIE if any anchor contradicts the claim. Pass the full timeline as anchors and an unrelated entry can spuriously contradict a true claim. Verify each claim against the entry it concerns; when you lack an explicit mapping, retrieve the relevant entry by vector similarity first — the pattern in the RAG Answer Gate recipe.

Step 3: A conclusion the matched entry supports

SELECT JSON_UNQUOTE(JSON_EXTRACT(NLI_VERIFY('nli-default', c.claim,
         JSON_ARRAY(t.event)), '$.verdict')) AS verdict
FROM recipe_incident_claims c
JOIN recipe_incident_timeline t ON c.timeline_id = t.id
WHERE c.id = 1;

Expected: TRUTH — the 02:14 entry entails it directly.

Step 4: A root cause the timeline does NOT establish

SELECT JSON_UNQUOTE(JSON_EXTRACT(NLI_VERIFY('nli-default', c.claim,
         JSON_ARRAY(t.event)), '$.verdict')) AS verdict
FROM recipe_incident_claims c
JOIN recipe_incident_timeline t ON c.timeline_id = t.id
WHERE c.id = 2;

Expected: UNVERIFIED_PLAUSIBLE — the matched entry records a failover, not a config change. The claim isn't contradicted, it's ungrounded: the most common way a postmortem goes wrong is asserting a cause the evidence doesn't establish. Treat UNVERIFIED_PLAUSIBLE on a root-cause statement as "attach evidence or soften to a hypothesis."

Step 5: A reassurance that has no supporting entry

A blanket "no customer data was affected" with no timeline entry to back it is exactly what review should stop. Verified against an outage entry, the model reads it as contradicted:

SELECT JSON_UNQUOTE(JSON_EXTRACT(NLI_VERIFY('nli-default',
  'No customer data was affected.',
  JSON_ARRAY('At 02:14 the primary database failed over to the replica, triggering the outage.')),
  '$.verdict')) AS verdict;

Expected: LIE. The lesson isn't that data was affected — it's that a reassurance with no supporting evidence should not pass review on the author's word. Add a data-integrity check to the timeline, then verify against that entry.

Step 6: Sign-off gate — every claim classified

SELECT c.id, c.claim, t.ts,
       JSON_UNQUOTE(JSON_EXTRACT(NLI_VERIFY('nli-default', c.claim,
         JSON_ARRAY(t.event)), '$.verdict')) AS verdict
FROM recipe_incident_claims c
JOIN recipe_incident_timeline t ON c.timeline_id = t.id
ORDER BY c.id;

Expected: claim 1 TRUTH, claim 2 UNVERIFIED_PLAUSIBLE, claim 3 TRUTH. The report is cleared to sign off only once every claim is TRUTH — or every exception is downgraded to an explicit hypothesis.

Cleanup (Optional)

DROP TABLE IF EXISTS recipe_incident_timeline;
DROP TABLE IF EXISTS recipe_incident_claims;

Use it from your app

• Wire the Step 6 query into the postmortem template's "ready for review" check; block sign-off on any non-TRUTH claim until it's grounded or reworded.
• Keep the timeline as atomic, timestamped entries, and match each claim to the entry it concerns (by structure or vector retrieval).
• Why in-DB: the timeline often already lives in the same database as your operational data; grounding the narrative against it is a query.

Key Concepts Learned

• Postmortem failure is usually ungrounded assertion, not contradiction — which is why UNVERIFIED_PLAUSIBLE on a root cause is the verdict to act on.
• A blanket reassurance with no supporting entry can read as a LIE; the fix is to record the check that justifies it, then re-verify against that entry.
• Match each claim to the entry it concerns — the whole timeline as anchors lets an unrelated entry spuriously contradict (contradiction precedence).
• Gate sign-off on verdicts, but keep a human in the loop — the verdict says "grounded / not grounded in this text", not "true / false in the world."