Area 02 // RAGLesson 37 / 14015 min read
BM25 keyword search and hybrid search
BM25 is the classic keyword-ranking formula that finds documents sharing the query's exact words (great for codes, names and rare terms), and hybrid search combines it with vector search so you catch both exact matches and matches by meaning.
Think of it like this
Analogy
Two librarians help you find a book. The first, the keyword librarian, checks the index cards: if you say "E-4012", she finds every book containing exactly "E-4012" instantly, but if you say "card refused" and the book says "payment declined", she finds nothing. The second, the meaning librarian, understands what you mean and finds the "payment declined" book, but shrugs at "E-4012".
Hybrid search asks both and combines their suggestions.
The problem it solves
Pure vector search is great at meaning, but real users type product codes, error numbers, people's names, legal clause numbers and jargon. Embedding models often treat "E-4012" and "E-4021" as nearly identical, or miss a rare name entirely. Pure keyword search has the opposite problem: it misses paraphrases and synonyms.
Production RAG needs both kinds of matching. Hybrid search is now the standard default for that reason.
How it works: BM25 keyword search
- Tokenise documents and query into words (lower-cased, sometimes stemmed: "refunds" → "refund").
- Build an inverted index: for each word, the list of documents containing it. Lookups are fast.
- Score each document with BM25, adding up a score for each query word that appears:
- Rarer words count more (inverse document frequency): "E-4012" in 2 of 10,000 docs is very informative; "the" is not.
- More occurrences help, with saturation: the 2nd mention adds less than the 1st, the 20th almost nothing.
- Length normalisation: a short document that mentions the word is more focused than a huge one that mentions it once.
- Rank by score. That's keyword search, the engine behind Elasticsearch, OpenSearch, Postgres full-text search and most site search boxes.
How it works: hybrid search
- Run both searches for the query: BM25 (top 20–50) and vector search (top 20–50).
- Merge with reciprocal rank fusion (RRF): each document's fused score is the sum of
1 / (k + rank)over the lists it appears in (k is usually 60). Documents ranked high in either list rise; documents high in both rise most. - Alternative: weighted score blending (e.g. 0.3 × normalised BM25 + 0.7 × cosine). Needs score normalisation and tuning; RRF usually works well without.
- Rerank the fused list with a reranker and keep the best few for the prompt.
- Tune with evals: compare BM25-only, vector-only and hybrid on your questions (Recall@k).
See it in code
- Install
pip install rank-bm25- Needs
- Python 3.10+
BM25 finds the doc with the exact error code; the vector ranking (simulated here) finds the doc that matches the meaning; RRF brings both to the top.
from rank_bm25 import BM25Okapi
docs = [
"Error E-4012 means the payment card was declined by the bank.",
"Our offices are closed on public holidays.",
"If a card payment fails, the bank may have blocked it for security.",
"Update your billing address in account settings.",
]
bm25 = BM25Okapi([d.lower().split() for d in docs]) # keyword index
query = "why was my card refused with e-4012"
bm25_scores = bm25.get_scores(query.lower().split())
bm25_rank = sorted(range(len(docs)), key=lambda i: -bm25_scores[i])
# Pretend semantic (vector) ranking for the same query: good at meaning ("refused" ~ "fails"),
# but an error code like E-4012 means little to it.
vector_rank = [2, 3, 0, 1]
def rrf(rankings, k=60):
"""Reciprocal rank fusion: each list votes 1/(k + position). No score tuning needed."""
scores = {}
for ranking in rankings:
for pos, doc in enumerate(ranking):
scores[doc] = scores.get(doc, 0) + 1 / (k + pos + 1)
return sorted(scores, key=lambda d: -scores[d])
print("BM25 top 2:", bm25_rank[:2])
print("vector top 2:", vector_rank[:2])
print("hybrid top 2:", rrf([bm25_rank, vector_rank])[:2], "<- the exact code AND the meaning match")BM25 top 2: [0, 1]
vector top 2: [2, 3]
hybrid top 2: [0, 2] <- the exact code AND the meaning matchBM25 alone put an irrelevant doc (1) second; vector search alone missed the error-code doc in its top 2. Hybrid's top 2 are exactly the two useful documents. In Postgres you can get BM25-style ranking with full-text search (to_tsvector, ts_rank) next to pgvector, and engines like Elasticsearch, OpenSearch, Qdrant and Weaviate offer hybrid search built in.
A real example
Example
A pharmacy assistant that confused drug names
A pharmacy's internal assistant used vector search over drug information leaflets. Pharmacists asking about "hydroxyzine" sometimes got leaflets for "hydralazine": different drugs with similar-looking names and, to the embedding model, similar meanings ("medicine for…"). That's a dangerous kind of mistake.
Adding BM25 with exact-name matching and fusing with RRF put the correct leaflet first in every test case with a drug name, while natural questions ("what can I take for hay fever?") still worked through the vector side. They also added a rule: if the query contains a known drug name, results must contain that exact name.
The easy way
Start here
If your search engine or database supports hybrid search out of the box, turn it on. Otherwise run BM25 (rank-bm25 or Postgres full-text) and vector search separately, merge with the 10-line RRF function above, and compare against vector-only on 30 real questions.
When you grow: the production setup
Production search tunes analysers (tokenisation, stemming, synonyms) per language, boosts fields like titles, uses learned sparse models (e.g. SPLADE-style) as a smarter keyword side, fuses with RRF, reranks with a cross-encoder, and tracks retrieval metrics per query type (codes, names, natural questions).
Trade-offs
What you gain
- BM25 is fast, cheap, explainable and excellent for exact terms.
- Hybrid search catches both exact matches and paraphrases, improving recall across query types.
- RRF fusion needs no score calibration.
What it costs you
- Two retrieval systems to build and keep in sync.
- More candidates means more reranking work and latency.
- Keyword search needs language-specific tuning (stemming, synonyms).
When not to use it
- Queries are almost always natural-language paraphrases with no codes or names: vector search alone may be enough (measure first).
- Tiny corpora where any method finds everything.
Common mistakes
- Relying on vector search alone for domains full of codes, IDs and names.
- Adding BM25 and vector scores directly as if they were on the same scale.
- Forgetting to keep the keyword index updated when documents change.
- Skipping a reranker after fusion, so the final top few are still noisy.
- Not measuring each retriever separately to see which one is failing.
Interview questions
Try answering out loud first, then open the model answer.
Explain BM25.
Model answer
BM25 is a probabilistic ranking function for keyword search. For each query term it combines inverse document frequency (rare terms weigh more), a saturating term-frequency component (repeated occurrences give diminishing returns, controlled by k1), and document-length normalisation (controlled by b). A document's score is the sum over query terms. It's fast via inverted indexes and strong for exact and rare terms.
Why use hybrid search in RAG, and how do you fuse results?
Model answer
Dense vector retrieval captures semantics but misses exact identifiers and rare terms; BM25 does the opposite. Running both and fusing increases recall across query types. Reciprocal rank fusion sums 1/(k + rank) across lists, avoiding score-scale mismatches; weighted blending of normalised scores is an alternative that needs tuning. A cross-encoder reranker then orders the fused candidates.
Users search with product SKUs and get irrelevant results from your RAG system. What do you do?
Model answer
Check whether retrieval is pure vector search; embeddings handle SKUs poorly. Add keyword search (BM25 or an exact-match filter on a SKU field), fuse with RRF, and consider routing queries that contain an identifier to exact lookup first. Store SKUs as structured metadata, and add SKU queries to the eval set to verify the fix.
Quiz
Pick an answer to see whether it’s right and why.
0 of 6 answered
Key terms
- BM25
- The classic keyword-ranking formula. Scores documents by the query words they contain, favouring rare words, with saturation and length normalisation.
- Keyword search
- Search that matches the exact words of the query against an inverted index, ranked with formulas such as BM25.
- Hybrid search
- Running keyword search and vector search together and merging their results, to catch both exact terms and meaning.
- Reciprocal rank fusion (RRF)
- Merging ranked lists by giving each item 1 / (k + rank) points from every list it appears in. Needs no score calibration.
- Semantic search
- Search by meaning rather than exact words, usually by comparing embeddings.
Before and after
Progress is saved in this browser only.