Policy Wording Consistency with NLI

Objective

A help page, chatbot answer, or agent note paraphrases policy — and paraphrases drift. Check whether derived wording still says what the governing policy says: a faithful paraphrase, a reversed exclusion, or a benefit the policy never granted.

Step 1: A faithful paraphrase

curl -s -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{
    "model": "nli-default",
    "statement": "Annual subscribers may cancel their subscription at any time.",
    "truth_anchors": ["An annual subscription can be canceled at any time."],
    "closed_world": false,
    "contradiction_threshold": 0.6, "entailment_threshold": 0.6
  }' http://localhost:8080/v1/ai/nli/verify

Expected (verified on v2.1.1-ce): verdict TRUTH, entailment 1.00 — the rewording preserves the policy.

Step 2: A reversed exclusion

curl -s -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{
    "model": "nli-default",
    "statement": "Free accounts can export audit logs.",
    "truth_anchors": ["Free accounts cannot export audit logs."],
    "closed_world": false,
    "contradiction_threshold": 0.6, "entailment_threshold": 0.6
  }' http://localhost:8080/v1/ai/nli/verify

Expected (verified): verdict LIE, contradiction 1.00 — the help text grants what the policy denies.

Step 3: An invented benefit

curl -s -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{
    "model": "nli-default",
    "statement": "A premium account includes priority onboarding.",
    "truth_anchors": ["A premium account includes unlimited projects."],
    "closed_world": false,
    "contradiction_threshold": 0.6, "entailment_threshold": 0.6
  }' http://localhost:8080/v1/ai/nli/verify

Expected (verified): verdict UNVERIFIED_PLAUSIBLE, neutral 1.00 — unlimited projects establish nothing about onboarding. The claim needs its own evidence or removal.

Honest boundaries

Select the policy version that actually governs the account and time period before submitting — the classifier does not know effective dates. Conditions, exceptions, and numeric eligibility boundaries need typed checks outside NLI, and a TRUTH verdict is review assistance, not a legal or access-control decision.

Key Concepts Learned

• Text consistency is checkable; eligibility is not — keep the authoritative rules system in the loop.
• The three claim failure modes map cleanly: reversal → contradiction, invented benefit → neutral, faithful paraphrase → entailment.