How to Search by Meaning Instead of Keywords

The problem: a customer types "my internet connection drops every few minutes" into your help center's search box. You have an article that answers it, called "My Wi-Fi keeps disconnecting". The search returns nothing, because the two sentences don't share a single word. The customer opens a ticket.

We build SynapCores, and the last part of this page runs in it. Everything before that is plain SQL run on SQLite, plus Postgres and MySQL equivalents, so you can follow it without installing anything of ours.

Searching by meaning instead of keywords: semantic search vs keyword search

There are three levels of "find the right row":

Semantic search turns every piece of text into an embedding: a list of a few hundred numbers produced by a small model, arranged so that texts with similar meanings get similar numbers. You compute one for each row, and one for the query, then sort rows by how close they are. Closeness is usually cosine similarity: 1.0 means the same direction, values near 0 mean unrelated.

Keyword search is not the loser in this table. It's exact, cheap and easy to explain. You'll see below that it beats meaning on error codes, and that the best setup usually uses both.

When this is useful

• People search in their own words: a help center, an internal wiki, support tickets, product reviews, policies.
• The same thing has many names ("cancel", "unsubscribe", "stop paying", "close my account").
• You want "related articles" or "similar tickets" without hand-written synonym lists.
• You're building a rag chatbot and need to find the passages to give the model. That step is semantic search.

The data

Nine help-center articles, each with a category and a last-updated date. They're the six from our r/SQLv2 Lesson 1 plus three more: a double charge, a refund policy, and an error code.

CREATE TABLE help_articles (
  id         INTEGER PRIMARY KEY,
  category   TEXT,
  updated_on TEXT,
  title      TEXT,
  body       TEXT
);

INSERT INTO help_articles (id, category, updated_on, title, body) VALUES
  (1, 'account',      '2026-03-02', 'How do I reset my password?',          'Use "Forgot password" on the login page. The reset link expires after one hour.'),
  (2, 'connectivity', '2026-09-14', 'My Wi-Fi keeps disconnecting',         'Move the router away from the microwave and update its firmware.'),
  (3, 'billing',      '2025-11-20', 'How can I change my billing address?', 'Settings > Billing > Edit address.'),
  (4, 'app',          '2026-08-01', 'The app crashes when I open it',       'Update to the latest version, then clear the app cache.'),
  (5, 'data',         '2026-01-10', 'Can I export my data to CSV?',         'Yes: Reports > Export > CSV.'),
  (6, 'billing',      '2026-07-22', 'How do I cancel my subscription?',     'Settings > Plan > Cancel. You keep access until the period ends.'),
  (7, 'billing',      '2026-10-03', 'Why was I charged twice?',             'A second charge is usually a pending authorisation. It drops off within 5 business days.'),
  (8, 'billing',      '2024-06-30', 'How do I get a refund?',               'Refunds are issued to the original card within 30 days of purchase.'),
  (9, 'account',      '2026-05-18', 'Error E-4012 when signing in',         'E-4012 means your single sign-on session expired. Sign out of your identity provider and sign in again.');

Sections 1 and 2 were run on SQLite 3.43.2 (macOS), and the outputs are pasted from that run. The Postgres and MySQL versions are shown for reference; we haven't run them.

1. LIKE: the exact substring

SELECT id, title FROM help_articles
WHERE title LIKE '%internet%' OR body LIKE '%internet%';

(0 rows)

No article contains "internet". That's expected. The less obvious failure is a word in a different form:

SELECT id, title FROM help_articles
WHERE title LIKE '%disconnects%' OR body LIKE '%disconnects%';

(0 rows)

"disconnects" isn't a substring of "disconnecting". LIKE also can't rank: a row matches or it doesn't. And a leading % means the database can't use an ordinary index, so it reads every row.

2. Full-text search: words, stems and a ranking

Full-text search splits text into words, reduces each word to its stem ("disconnecting" and "disconnects" both become "disconnect"), indexes them, and ranks matches. SQLite has it built in as FTS5:

CREATE VIRTUAL TABLE help_fts USING fts5(
  title, body,
  content='help_articles', content_rowid='id',
  tokenize='porter'
);
INSERT INTO help_fts(help_fts) VALUES ('rebuild');

SELECT a.id, a.title, ROUND(bm25(help_fts), 2) AS bm25
FROM help_fts JOIN help_articles a ON a.id = help_fts.rowid
WHERE help_fts MATCH 'router disconnects'
ORDER BY bm25;

id  title                         bm25
--  ----------------------------  -----
2   My Wi-Fi keeps disconnecting  -3.56

Stemming fixed the word-form problem. (bm25() in SQLite is negative, and lower is better, so ORDER BY bm25 puts the best match first.)

It also finds exact codes, which matters later:

SELECT a.id, a.title, ROUND(bm25(help_fts), 2) AS bm25
FROM help_fts JOIN help_articles a ON a.id = help_fts.rowid
WHERE help_fts MATCH '"E-4012"'
ORDER BY bm25;

id  title                         bm25
--  ----------------------------  -----
9   Error E-4012 when signing in  -2.11

Now the customer's actual sentence. By default, FTS5 requires every word:

SELECT a.id, a.title
FROM help_fts JOIN help_articles a ON a.id = help_fts.rowid
WHERE help_fts MATCH 'internet connection drops';

(0 rows)

Loosen it to any word, and you get an answer. The wrong one:

SELECT a.id, a.title, ROUND(bm25(help_fts), 2) AS bm25
FROM help_fts JOIN help_articles a ON a.id = help_fts.rowid
WHERE help_fts MATCH 'internet OR connection OR drops'
ORDER BY bm25;

id  title                     bm25
--  ------------------------  -----
7   Why was I charged twice?  -1.62

"A pending authorisation … drops off within 5 business days." The word matched; the meaning didn't. The same thing happens with "stop paying": zero rows, because no article uses either word. Full-text search knows words, not what they're about.

Full-text search does combine cleanly with filters, which is the other half of real search:

SELECT a.id, a.category, a.updated_on, a.title
FROM help_fts JOIN help_articles a ON a.id = help_fts.rowid
WHERE help_fts MATCH 'charge OR refund'
  AND a.category = 'billing'
  AND a.updated_on >= '2026-01-01';

id  category  updated_on  title
--  --------  ----------  ------------------------
7   billing   2026-10-03  Why was I charged twice?

The refund article matched "refund" but was dropped by the date filter, because it was last updated in 2024.

In Postgres (not run here), the same idea is a tsvector column with a GIN index:

ALTER TABLE help_articles
  ADD COLUMN tsv tsvector
  GENERATED ALWAYS AS (to_tsvector('english', title || ' ' || body)) STORED;
CREATE INDEX help_articles_tsv ON help_articles USING GIN (tsv);

SELECT id, title, ts_rank(tsv, q) AS rank
FROM help_articles, websearch_to_tsquery('english', 'router disconnects') AS q
WHERE tsv @@ q
ORDER BY rank DESC;

In MySQL (not run here), it's a FULLTEXT index and MATCH … AGAINST. MySQL doesn't stem words, so "disconnects" won't match "disconnecting" there:

ALTER TABLE help_articles ADD FULLTEXT ft_help (title, body);

SELECT id, title,
       MATCH(title, body) AGAINST ('router disconnects' IN NATURAL LANGUAGE MODE) AS score
FROM help_articles
WHERE MATCH(title, body) AGAINST ('router disconnects' IN NATURAL LANGUAGE MODE)
ORDER BY score DESC;

If your users search with the words your articles use, stop here. Postgres full-text search, with a synonym list for the handful of terms people mix up, is a good search box, and you already run it.

3. Search by meaning: embeddings and cosine similarity

Semantic search needs three things:

1. An embedding model. A small one is enough for search. Common choices output 384 or 768 numbers per text.
2. A vector column on each row, holding the embedding of that row's text. Embed the title and body together; a title alone often carries too little meaning.
3. A query that embeds the search text with the same model and sorts rows by similarity.

The design choice is where the embedding is computed, because that decides how many systems you run and whether your filters live next to your vectors. There are three common answers.

Postgres + pgvector (not run here). The database stores and compares vectors; your application computes them, with a local library such as sentence-transformers or a hosted embeddings API:

CREATE EXTENSION IF NOT EXISTS vector;
ALTER TABLE help_articles ADD COLUMN embedding vector(384);
-- your app: for each row, embedding = model.encode(title + '. ' + body), then UPDATE

-- $1 is the query's embedding, computed by your app with the same model
SELECT id, title, 1 - (embedding <=> $1) AS similarity
FROM help_articles
WHERE category = 'billing'
ORDER BY embedding <=> $1
LIMIT 3;

CREATE INDEX ON help_articles USING hnsw (embedding vector_cosine_ops);

This is a strong, common setup. Two things to plan for: the app has to re-embed a row whenever its text changes, and with an approximate index plus a selective WHERE, the index can hand back fewer than LIMIT rows (recent pgvector versions add iterative index scans for this; check the docs for your version).

A dedicated vector database (Pinecone, Qdrant, Weaviate, Milvus and others). Vectors and a copy of the metadata you filter on live in a separate service built for large vector workloads. You keep it in sync with your main database, and you join its results back to your rows in application code.

Embeddings computed in the database (SynapCores, below). The model runs inside the engine, so EMBED('the query') is a SQL expression. The row text, the embedding, the filters and the similarity are in one statement, and nothing is sent to an external embedding service.

How the options compare

When full-text search is enough: your users and your documents use the same vocabulary, or a short synonym list covers the gap. It's in the database you already run, and it explains itself.

When pgvector is enough: you're on Postgres, you're fine adding an embedding step to your write path, and you want a mature approximate index. For most teams that already run Postgres, this is the default answer, and nothing on this page argues against it.

When a dedicated vector database is worth it: vectors are the main workload, the corpus is very large, and a separate service with its own scaling is easier for you than tuning Postgres.

What in-database EMBED changes is where the embedding step lives, not what similarity can find. There's no embedding job to run or keep in sync, and the query is plain SQL. We haven't benchmarked it against pgvector or any vector database, and this page makes no claim that it's faster or more accurate.

Hybrid search: keywords plus meaning plus filters

Real search boxes get both kinds of query. "I want to stop paying" needs meaning. "E-4012" needs the exact token: to an embedding model, E-4012 and E-4013 look almost the same, so meaning alone can't tell a code you have from one you don't. And almost every query has filters that aren't text at all: this customer's plan, this tenant, published, updated in the last year.

Hybrid search runs a keyword score and a meaning score for the same query and merges them, inside the filters. Two common ways to merge:

• Reciprocal rank fusion (RRF). Take the keyword ranking and the meaning ranking, give each row 1 / (60 + rank) from each list, and add them. It needs no tuning, because it ignores the raw scores, which aren't comparable between BM25 and cosine similarity. 60 is the constant commonly used with RRF.
• A weighted sum. Add a fixed boost to the similarity when an exact keyword matches. Simpler, but the weight is a judgement call you should test on real queries.

Whichever you pick, apply the structured filters before ranking, not after. If you rank the top 10 by meaning and then drop the stale or wrong-tenant ones, you can end up with 2 results, or none.

Search by meaning in one SQL statement (SynapCores)

This part runs on SynapCores 2.3.2-ce. The embedding model (all-minilm, 384 dimensions) runs inside the engine. There's no API key and no external embedding service.

curl -fsSL https://get.synapcores.com | SYNAPCORES_VERSION=v2.3.2-ce sh

Create the table with a vector column, and load the same nine rows. VECTOR(384) must match the model's output size. If it doesn't, the write fails with a clear error. On a VECTOR(768) column, 2.3.2 refuses the UPDATE with "VECTOR(768) column 'embedding' rejected 384-dimension value. Embedding-provider mismatch …".

CREATE TABLE help_articles (
  id         INTEGER PRIMARY KEY,
  category   TEXT,
  updated_on TEXT,
  title      TEXT,
  body       TEXT,
  embedding  VECTOR(384)
);

-- then the same INSERT as in "The data" above (nine rows, no embedding yet)

Let the database compute the meanings. One statement embeds every row, title and body together:

UPDATE help_articles SET embedding = EMBED(title || '. ' || body);

rows_affected
-------------
9

Keyword search, for the record:

SELECT id, title FROM help_articles
WHERE title LIKE '%internet%' OR body LIKE '%internet%';

(0 rows)

Search by meaning. Embed the customer's sentence and sort by cosine similarity:

SELECT id, title,
       ROUND(COSINE_SIMILARITY(embedding, EMBED('my internet connection drops every few minutes')), 2) AS score
FROM help_articles
ORDER BY score DESC
LIMIT 3;

id  title                           score
--  ------------------------------  -----
2   My Wi-Fi keeps disconnecting    0.43
7   Why was I charged twice?        0.19
4   The app crashes when I open it  0.18

Compute the similarity in the SELECT list and sort by its alias, so the score is shown next to each row.

Meaning plus structured filters, one query. This is the part that needs embeddings in the database. A customer on the billing page types "I was billed two times this month". Search only billing articles updated this year:

SELECT id, title, updated_on,
       ROUND(COSINE_SIMILARITY(embedding, EMBED('I was billed two times this month')), 2) AS score
FROM help_articles
WHERE category = 'billing'
  AND updated_on >= '2026-01-01'
ORDER BY score DESC
LIMIT 3;

id  title                             updated_on  score
--  --------------------------------  ----------  -----
7   Why was I charged twice?          2026-10-03  0.7
6   How do I cancel my subscription?  2026-07-22  0.39

The query text, the embedding of the query, the category filter, the date filter and the ranking are one statement against one copy of the data. There's no vector store to query first, no ids to pass back, and no top 10 that shrinks to 2 after filtering. The WHERE columns can use ordinary indexes, which also means fewer rows to compare.

Hybrid: an exact keyword plus meaning, inside the filters. First, meaning alone, for the error code:

SELECT id, title,
       ROUND(COSINE_SIMILARITY(embedding, EMBED('E-4012')), 2) AS score
FROM help_articles
ORDER BY score DESC
LIMIT 3;

id  title                         score
--  ----------------------------  -----
9   Error E-4012 when signing in  0.54
7   Why was I charged twice?      0.25
1   How do I reset my password?   0.16

That looks fine: the right article is first, by a wide margin. The problem shows up with a code you have no article for. Search for E-4013:

SELECT id, title,
       ROUND(COSINE_SIMILARITY(embedding, EMBED('E-4013')), 3) AS score
FROM help_articles
ORDER BY score DESC
LIMIT 3;

id  title                         score
--  ----------------------------  -----
9   Error E-4012 when signing in  0.474
7   Why was I charged twice?      0.195
8   How do I get a refund?        0.179

No article mentions E-4013, yet the E-4012 article comes back at 0.47, well clear of the rest. A keyword filter (LIKE '%E-4013%') returns nothing, which is the right answer. Meaning search can't tell close codes apart.

Add a keyword predicate as a boost, and keep the filters:

SELECT id, title,
       ROUND(meaning, 2) AS meaning,
       keyword,
       ROUND(meaning + keyword, 2) AS hybrid
FROM (
  SELECT id, title,
         COSINE_SIMILARITY(embedding, EMBED('E-4012')) AS meaning,
         CASE WHEN title LIKE '%E-4012%' OR body LIKE '%E-4012%' THEN 0.3 ELSE 0.0 END AS keyword
  FROM help_articles
  WHERE updated_on >= '2025-01-01'
) t
ORDER BY hybrid DESC
LIMIT 3;

id  title                         meaning  keyword  hybrid
--  ----------------------------  -------  -------  ------
9   Error E-4012 when signing in  0.54     0.3      0.84
7   Why was I charged twice?      0.25     0.0      0.25
1   How do I reset my password?   0.16     0.0      0.16

The 0.3 boost is a choice for this example, not a recommended value. In an application, your code picks the exact tokens out of the query (anything that looks like a code, a SKU or a quoted phrase) and passes them as the keyword parameter. Ordinary queries get no boost and rank by meaning alone.

A threshold instead of top-N. A search box that always returns three results will return three results for a question you have no article for. A cut-off lets it say "no match":

SELECT id, title,
       ROUND(COSINE_SIMILARITY(embedding, EMBED('forgot my login')), 2) AS score
FROM help_articles
WHERE COSINE_SIMILARITY(embedding, EMBED('forgot my login')) > 0.5;

id  title                        score
--  ---------------------------  -----
1   How do I reset my password?  0.63

SELECT id, title,
       ROUND(COSINE_SIMILARITY(embedding, EMBED('do you ship to Canada?')), 2) AS score
FROM help_articles
WHERE COSINE_SIMILARITY(embedding, EMBED('do you ship to Canada?')) > 0.5;

(0 rows)

Keep embeddings fresh. When an article's text changes, re-embed that row in the same statement. An embedding of old text is worse than none, because it still matches:

UPDATE help_articles
SET body      = 'A second charge is usually a pending authorisation. It drops off within 3 business days.',
    updated_on = '2026-10-25',
    embedding = EMBED('Why was I charged twice?. A second charge is usually a pending authorisation. It drops off within 3 business days.')
WHERE id = 7;

rows_affected
-------------
1

Clean up when you're done:

DROP TABLE IF EXISTS help_articles;

Thresholds, and why similarity isn't truth

Similarity scores aren't calibrated. On this page, the right article scored 0.43 for the internet query, with 0.19 next, and 0.63 for "forgot my login", with 0.36 next (title + body, 2.3.2). In Lesson 1 (title-only embeddings, 2.3.1), the same kind of query scored 0.56 for the right article, with 0.26 next. Scores move with the model, the length of what you embed, and the domain. So:

• Collect 20 to 50 real queries from your search log, with the article that should win.
• For each, note the right answer's score and the best wrong answer's score.
• Put the threshold in the gap between them. If there's no gap, the threshold can't save you: fix what you embed (title plus body, not title alone) or add keyword and filter signals.
• Re-check the threshold whenever you change the model or what you embed.

Similarity means "about the same thing", not "says the same thing". A sentence and its opposite are about the same thing. In our test for How to Check If an AI Answer Is Actually Backed by Your Documents, "Refunds arrive within 24 hours" scored 0.779 similarity to a policy that says refunds take 5 to 10 business days. For a search box, that's fine: the policy is the right page to show. For a chatbot that writes an answer from the top result, it's how a wrong answer gets a confident source. Use similarity to find candidates, then check the claim against the text. That article shows how.

On our nine articles, 'change my delivery address' scores 0.63 against 'change my billing address', above the threshold, and that article doesn't answer it.

A checklist before you ship search by meaning

• Prove keyword search fails first. Run the real queries from your search log through full-text search; if it already works, you may not need embeddings.
• Embed the title and body together, with one model, and store the model name next to the vectors.
• Match the vector column size to the model's output.
• Re-embed a row whenever its text changes.
• Apply structured filters (tenant, category, date, permissions) in the same query, before ranking.
• Keep an exact-match path for codes, SKUs and names: hybrid, not either/or.
• Set the threshold from a labelled set of real queries, and re-check it on every model change.
• Treat the top result as "on topic", never as "correct".

Where to go next

• Next: How to Find Duplicate Customer Records, Even With TyposComing soon: the same similarity, used to match messy records, with SQL blocking keys so you don't compare every row with every other.
• Related: How to Check If an AI Answer Is Actually Backed by Your Documents: why a high similarity score isn't a fact check.
• Related: How to Build a Support Chatbot That Only Answers From Your Help Articles (With Sources)Coming soon: this search, plus citations and an "I don't know" gate.
• Before this: How to Ask Your Database Questions in Plain EnglishComing soon and Which Technique Solves My Problem?
• Run it end to end: the semantic search recipe (about 8 minutes) and r/SQLv2 Lesson 1, the beginner version of this page.

Try it: download SynapCores Community Edition, load the nine articles above, and search them in your own words. Then run the same queries through your current search and compare.