How to Make a Chatbot Remember Previous Conversations

The problem: your chatbot forgets everything when the conversation ends, so you add a memory store. A month later it tells a user something that stopped being true weeks ago, or it "remembers" one customer's allergy as another's. The memory is there. It's just wrong.

We build SynapCores, and the last section uses it. Everything before that works with any stack, and the examples name open-source models.

How a chatbot remembers: short-term vs long-term memory

Short-term memory is mostly a prompt-management problem: trimming, summarising, keeping the important turns. This article is about long-term memory.

Episodic vs semantic memory

Long-term memory usually holds two kinds of thing:

• Episodic memory is what happened. "On Oct 3 the user asked about refunds and we said 5 to 10 business days." Episodes are append-only. A new episode never makes an old one false.
• Semantic memory is what is true now. "The user lives in Denver." "The account is on the Team plan." When the world changes, a new semantic fact should replace an old one.

Some systems also keep procedural memory: how to do things, such as "this user wants answers as bullet points" or "run the tests before opening a PR". It behaves like semantic memory, because a newer instruction replaces an older one.

The split matters because corruption is almost always a semantic-memory problem. Appending an episode can't make the store wrong. Replacing a fact can.

How to give an AI agent long-term memory

Most chatbot and agent memory, whatever the product, is the same loop:

 conversation ──► extract facts ──► WRITE RULE ──► store
                                       │
                         add / replace / skip / review
                                                    │
 next turn ◄── put in prompt ◄── recall top-k ◄─────┘

1. Extract. An LLM or a rule pulls short facts out of the conversation ("prefers email", "moved to Austin").
2. Write. Something decides whether the fact is new, a duplicate, or an update to an existing memory.
3. Recall. At the start of a turn, find the memories most related to the current message, usually with embeddings and similarity search.
4. Use. Add them to the prompt.

Frameworks such as Mem0 and LangGraph's long-term store handle some or all of these steps for you, each in its own way.

When something simple is enough

Be honest about what you need before adding machinery.

• The facts have fixed keys. Plan, timezone, language, preferred name. Use a key-value store or a JSON profile per user. "Replace" means overwrite the key. There is nothing to get wrong about which memory to replace, and no model is needed.
• You only need episodes. A log of past conversations with search over it. Append-only can't be corrupted by updates.
• You're already on a framework. If your agent runs on LangGraph or uses Mem0, start with their memory. Add a stricter write check later if you find stale or wrongly merged facts.
• A person reviews memory. If someone approves each stored fact, a simple store plus a review screen is enough.

You need more than that when facts arrive as free text, from several sources, and nobody checks them before they're stored. That is where memory starts to rot.

How memory gets corrupted

Three things go wrong, and they pull in opposite directions.

1. Stale facts stay. "Maya moved to Austin last month" arrives. The store already has "Maya lives in Denver, Colorado." If the write rule only adds, both are stored, and recall can return either. The agent now has two cities for one person.

2. Contradictions are written blindly. A scraper or a second agent writes "the customer's office is in Chicago" without checking what's already known. Whether the new or old fact is right, nobody saw the conflict.

3. Look-alikes get overwritten. The fix for problem 1 is "replace the old fact". But which one? Memories that look alike are often about different things:

Replace any of those and the agent loses a true fact. That's worse than keeping a stale one: a stale fact can be noticed and fixed, but a deleted one is simply gone. (Examples from our internal, synthetic test set.)

Why similarity alone can't decide

The obvious write rule is "if the new fact is very similar to a stored one, replace it". Similarity measures topic, not meaning, so it fails both ways:

• Real updates often aren't that similar. "Moved to Austin" and "lives in Denver" share a person, not many words. A high threshold such as 0.95 catches almost none of them. In our internal tests, a similarity-only upsert at 0.95 missed nearly every real update.
• Look-alikes often are similar. The return and outbound flights score higher than many real updates do. Lower the threshold to catch updates, and you start replacing siblings.

Why a contradiction model alone can't decide either

The next idea is to ask a model whether the new fact contradicts the old one. An NLI (natural language inference) model does exactly that: given two sentences, it scores entailment, contradiction and neutral.

It's good at real conflicts. It's bad at saying "these are unrelated". In recipe 036 (re-run on SynapCores 2.3.2, cross-encoder/nli-deberta-v3-small), "The customer is learning to play the cello." scored 0.996 to 1.000 contradiction against every one of four unrelated memories, including "The customer's office is in Denver." The model was trained on sentence pairs meant to be compared, so it never learned to say "different topic".

Let that model decide which memories to overwrite, and it deletes unrelated facts. In our internal tests, NLI with no similarity gate often overwrote good memories that had nothing to do with the new fact.

The fix: gate every write

Use each tool for what it's good at. Similarity decides what's related. NLI decides whether a related memory is contradicted. Then be strict about what you delete.

For each incoming semantic fact:

1. Recall candidates. Find the 20 most similar stored memories.
2. Similarity floor. Keep only candidates with similarity above 0.60. This removes the cello-style pairs before the NLI model sees them.
3. Contradiction check. Score each remaining candidate with NLI (stored memory as premise, new fact as hypothesis). Keep only those with contradiction above 0.95.
4. Replace only the closest match. If any candidate passes, replace the single most similar one. Never replace every memory that passes: one broad fact ("Maya downgraded to the Free plan") can look like a contradiction of many unrelated memories about the same person.
5. Keep the history. Store the new fact with a pointer to the memory it replaced, so a bad replacement can be undone.
6. Everything else is stored as new, or queued for review if you want a person to look at near-misses.

These thresholds are strict on purpose: a false "no conflict" leaves a stale fact (bad), a false "conflict" deletes a good one (worse). They came from tuning on our own data; treat them as a starting point, not a constant.

The gate in Python (any stack)

This uses open-source models: a small embedding model and the same NLI model as the SQL version below. Swap the store for whatever you use.

# pip install "sentence-transformers==5.1.2"
from sentence_transformers import SentenceTransformer, CrossEncoder, util

emb = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
nli = CrossEncoder("cross-encoder/nli-deberta-v3-small")
LABELS = ["contradiction", "entailment", "neutral"]   # the model's id2label order
SIM_FLOOR, CONTRA_MIN = 0.60, 0.95

memories = [
    {"id": 1, "text": "The customer's office is in Denver."},
    {"id": 2, "text": "The customer prefers to be contacted by email, never by phone."},
    {"id": 3, "text": "The customer manages procurement for a mid-size logistics company."},
]

def pick_memory_to_replace(new_fact, memories, top_k=20):
    """Return the id of the one memory to replace, or None."""
    new_vec = emb.encode(new_fact)
    scored = [(util.cos_sim(new_vec, emb.encode(m["text"])).item(), m) for m in memories]
    related = [(s, m) for s, m in sorted(scored, key=lambda x: -x[0])[:top_k] if s > SIM_FLOOR]
    for sim, m in related:                          # most similar first
        probs = nli.predict([(m["text"], new_fact)], apply_softmax=True)[0]
        contra = float(probs[LABELS.index("contradiction")])
        print(f"  sim={sim:.3f} contra={contra:.3f}  {m['text']}")
        if contra > CONTRA_MIN:
            return m["id"]                          # closest contradicted memory only
    return None

for fact in ["The customer's office is in Chicago.",
             "The customer is learning to play the cello."]:
    print(fact, "->", pick_memory_to_replace(fact, memories))

Output (Python 3.9, sentence-transformers 5.1.2, transformers 4.57.6, torch 2.8.0):

  sim=0.740 contra=0.999  The customer's office is in Denver.
The customer's office is in Chicago. -> 1
The customer is learning to play the cello. -> None

The cello fact never reaches the NLI model: nothing clears the 0.60 floor, so it's stored as new.

How well it held up (and where it didn't)

We tested this gate internally on synthetic memories and incoming facts that we wrote ourselves, not on real transcripts. We haven't published those results, so this section describes what we saw without numbers.

• Strict settings trade recall for safety. In our internal tests, the strict gate avoided wrongly overwriting good facts, at the cost of missing some real updates. Looser settings caught more updates and started overwriting look-alikes.
• A clean test run is not "never". A small test set can't prove a gate never makes a wrong replacement. Keep the history pointer (step 5) so you can undo one.
• Missed updates still hurt. An update the gate misses is stored next to the old fact, so recall can still surface the stale value.
• The safe zone is narrow. Small changes to the contradiction threshold changed the results, so measure before you move it.
• Real memory is messier. Real memories are longer and pack several claims into one sentence, which makes NLI less reliable. Expect worse results on real transcripts. We haven't tested those yet.

The biggest weakness: updates phrased as events. The NLI model reads "X changed to Y" as compatible with "X is the old value":

Both pairs are closely related, and both are real updates, but the gate misses both. Two ideas we haven't measured yet: rewrite incoming events as plain statements of the current state before the check ("the staging database runs PostgreSQL 16"), or give memories attribute keys so an update to db_version replaces the old db_version directly. Until then, route missed updates like these to review.

The same gate in one SQL statement (SynapCores)

In SynapCores, memory, embeddings and the NLI model run inside the database, so the whole write path is one statement. MEMORY_RECALL is a table function that returns similar memories with a similarity column, NLI_CLASSIFY scores each pair, and the outer SELECT stores the new fact and forgets the one it replaces. If no memory passes both checks, the inner query returns no row and nothing is replaced. NLI is off by default; recipe 036 shows how to enable it.

SELECT MEMORY_STORE('recipe_memcheck_ns', 'The customer''s office is in Chicago.',
                    json_object('supersedes', v.id)) AS new_id,
       MEMORY_FORGET('recipe_memcheck_ns', v.id) AS forgotten
FROM (
  SELECT id, similarity FROM (
    SELECT id, similarity,
           CAST(JSON_EXTRACT(NLI_CLASSIFY('nli-default', content,
                'The customer''s office is in Chicago.'),
                '$.scores.contradiction') AS DOUBLE) AS con
    FROM MEMORY_RECALL('recipe_memcheck_ns', 'The customer''s office is in Chicago.', 20)
    WHERE similarity > 0.60
  ) t
  WHERE con > 0.95
  ORDER BY similarity DESC
  LIMIT 1
) v;

This statement uses this article's strict settings, because it replaces a memory with nobody watching. Recipe 036 itself triages a whole batch with a looser 0.45 similarity floor and sends conflicts to a review queue instead of replacing them.

Output on SynapCores 2.3.2, with the four recipe 036 memories stored in recipe_memcheck_ns. The inner query picks the Denver memory (similarity 0.740, contradiction 0.999), and the statement returns one row:

new_id                 forgotten
---------------------  ---------
mem_1m5sv92ns_8qbJ3F   true

Recalling the namespace afterwards shows "The customer's office is in Chicago." with metadata {"supersedes":"mem_1m5sv92hk_fqOzuc"} (the Denver memory's id), and the other three memories untouched. The same statement with the cello fact returns no rows: its closest memory is at similarity 0.235, below the 0.60 floor, so nothing is forgotten.

What it doesn't do: it doesn't fix the event-phrasing weakness above, and it doesn't decide which source to trust. If the stored memory was wrong to begin with, the correct new fact is the one that looks like a "conflict".

If you use MEMORY_UPSERT instead, note that on its own it decides on similarity, with no contradiction check (the similarity-only rule from earlier). Pass numeric options with CAST(x AS DOUBLE).

SELECT MEMORY_UPSERT('recipe_memcheck_ns',
         'The customer prefers to be contacted by email.',
         json_object('policy', 'replace',
                     'similarity_threshold', CAST(0.85 AS DOUBLE))) AS action;

action
------
ADD

On 2.3.2 the CAST is still needed: with a bare 0.85 the call fails with "MEMORY_UPSERT option 'similarity_threshold' must be a number". It returns ADD here because the closest stored memory, "The customer prefers to be contacted by email, never by phone.", is at similarity 0.80, below the 0.85 threshold, so both are now stored side by side.

A checklist before you ship agent memory

• Split memory into episodes (append-only) and facts (replaceable). Only facts need a write rule.
• Use fixed keys wherever you can. A keyed fact needs no model to update.
• For free-text facts, gate writes: similarity floor, then a confident contradiction, then replace the closest match only.
• Keep history. Never delete without a pointer to what replaced it.
• Send near-misses to review, and log every replacement so you can audit it.
• Measure on your own memories before trusting any threshold, including ours. Re-check when you change the embedding model, the NLI model or the engine version.

Where to go next

• Next: How to Let Several AI Agents Share One MemoryComing soon
• Related: How to Check If an AI Answer Is Actually Backed by Your Documents covers the same similarity-vs-NLI trap for generated answers.
• Related: How to Check If Two Sentences Mean the Same Thing (or Contradict Each Other)Coming soon

Try it: download SynapCores Community Edition and run the agent memory contradiction-check recipe end to end.