Objective
On a voice call, every second of latency is dead air — and most callers ask the same handful of questions in slightly different words. An exact-match cache misses all the paraphrases. A semantic cache hits when the meaning matches: if you've already answered "what time do you close?", you instantly reuse it for "how late are you open?" — no LLM round trip, no dead air, no token cost. Here you'll build a semantic answer cache with a vector lookup and a hit threshold. Your STT/TTS and telephony live outside; the cache is the database. See Use it from your agent for wiring it into any voice stack.
Step 1: Create the semantic answer cache
Each row is a previously answered question, its answer, an embedding, and a hit counter.
CREATE TABLE IF NOT EXISTS recipe_voice_cache (
cache_id INTEGER PRIMARY KEY,
question TEXT,
answer TEXT,
hits INTEGER DEFAULT 0,
embedding VECTOR(384)
);
Step 2: Warm the cache with common Q&A
The questions a phone line answers all day — store the canonical answer once.
INSERT INTO recipe_voice_cache (cache_id, question, answer) VALUES
(1,'What are your hours?','We''re open 9am to 8pm Monday through Saturday, closed Sundays.'),
(2,'Where can I park?','There''s free customer parking in the lot behind the building.'),
(3,'What is your return policy?','You can return items within 14 days with a receipt for a full refund.'),
(4,'Do you offer delivery?','Yes, same-day delivery before 2pm within 10 miles.'),
(5,'What payment methods do you take?','We accept all major cards, mobile wallets, and cash.');
Step 3: Embed the cached questions
The embedding model runs in-database; this is the index the cache is looked up against.
UPDATE recipe_voice_cache SET embedding = EMBED(question);
Step 4: Look up a paraphrased question in the cache
The caller asks "how late are you open tonight?"; find the closest cached question by meaning.
SELECT cache_id, question, answer,
COSINE_SIMILARITY(embedding, EMBED('how late are you open tonight?')) AS similarity
FROM recipe_voice_cache
ORDER BY similarity DESC
LIMIT 1;
Step 5: Decide hit vs. miss with a threshold
Serve the cached answer only on a strong semantic match; otherwise fall through to the live LLM path.
SELECT answer, similarity,
CASE WHEN similarity >= 0.55 THEN 'CACHE_HIT' ELSE 'MISS_CALL_LLM' END AS decision
FROM (
SELECT answer,
COSINE_SIMILARITY(embedding, EMBED('what time do you shut tonight?')) AS similarity
FROM recipe_voice_cache
ORDER BY similarity DESC
LIMIT 1
);
Step 6: Count the hit (so you can measure cache value)
On a hit, increment the counter for the matched entry to track how much latency/cost the cache saves. We first pin the matched entry into a one-row table (projecting the similarity we order by), then increment by that id.
CREATE TABLE IF NOT EXISTS recipe_voice_cache_hit (id INTEGER PRIMARY KEY, cache_id INTEGER, similarity DOUBLE);
INSERT INTO recipe_voice_cache_hit (id, cache_id, similarity)
SELECT 1, cache_id,
COSINE_SIMILARITY(embedding, EMBED('what time do you shut tonight?')) AS similarity
FROM recipe_voice_cache
ORDER BY similarity DESC
LIMIT 1;
UPDATE recipe_voice_cache
SET hits = hits + 1
WHERE cache_id = (SELECT cache_id FROM recipe_voice_cache_hit LIMIT 1);