Objective
A voice agent that forgets what the caller said two turns ago feels broken — and a long call quickly overruns the context the model can hold. Voice is just text once it's transcribed, so the same memory pattern applies: store every transcribed turn, recall older ones by meaning, and maintain a rolling summary so the agent keeps the thread on a 10-minute call. Your STT/TTS and telephony (Vapi, LiveKit, Twilio, Pipecat, Retell) live outside the database; the memory is the database. See Use it from your agent for wiring it into any voice stack.
Step 1: Create the voice turn store
One row per transcribed utterance, tagged by call and speaker, embedded for recall.
CREATE TABLE IF NOT EXISTS recipe_voice_turns (
turn_id INTEGER PRIMARY KEY,
call_id TEXT,
speaker TEXT, -- 'caller' or 'agent'
transcript TEXT,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
embedding VECTOR(384)
);
Step 2: Create the rolling call summary store
One compact summary per call, so older turns can be dropped from the live context.
CREATE TABLE IF NOT EXISTS recipe_voice_summary (
call_id TEXT PRIMARY KEY,
summary TEXT,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
Step 3: Record a transcribed conversation
What STT would hand you, turn by turn, during a support call.
INSERT INTO recipe_voice_turns (turn_id, call_id, speaker, transcript) VALUES
(1,'call-42','caller','Hi, my internet has been dropping every evening this week.'),
(2,'call-42','agent','I''m sorry to hear that. Is it all devices or just one?'),
(3,'call-42','caller','All of them, and it''s worst around 8pm.'),
(4,'call-42','agent','Got it. Have you restarted the router recently?'),
(5,'call-42','caller','Yesterday, but it didn''t help. Also I work from home so this is urgent.'),
(6,'call-42','agent','Understood, I''ll prioritize this. What''s your account number?');
Step 4: Embed the turns
The embedding model runs in-database, so older turns are recallable by meaning, not just position.
UPDATE recipe_voice_turns SET embedding = EMBED(transcript);
Step 5: Recall the relevant earlier turns mid-call
The caller circles back to "when does it happen?"; pull the turns that answer it without replaying the whole call.
SELECT turn_id, speaker, transcript,
COSINE_SIMILARITY(embedding, EMBED('when and how often does the connection drop?')) AS relevance
FROM recipe_voice_turns
WHERE call_id = 'call-42'
ORDER BY relevance DESC
LIMIT 3;
Step 6: Maintain a rolling summary of the call
Compress the call so far into two sentences the agent can carry cheaply on every turn.
INSERT INTO recipe_voice_summary (call_id, summary)
SELECT 'call-42',
GENERATE('Summarize this support call in 2 sentences (problem, key details, urgency): ' ||
(SELECT GROUP_CONCAT(speaker || ': ' || transcript, '\n') FROM recipe_voice_turns WHERE call_id='call-42'));