Pure semantic search fails on exact phrases, product names, and rare terms. Pure keyword search fails on synonyms and conceptual queries. Hybrid search combines both, and consistently outperforms either approach alone on real-world retrieval benchmarks.
A user searches for “CVE-2024-1234” — a specific vulnerability ID. No embedding model will retrieve the right document based on meaning, because the ID itself has no semantic content. The same applies to: product codes, legal case numbers, gene names, version strings, and any other token that only matches lexically.
A user searches for “Python snake handling techniques”— an embedding model may return Python programming tutorials because the word “Python” dominates the embedding. Keyword search suffers the same ambiguity problem differently.
BM25 (Best Match 25) is the ranking function underlying most search engines. It scores documents based on term frequency (TF) — how often the query terms appear — normalized by document length and inverse document frequency (IDF) — how rare those terms are across the whole corpus.
You don't need to memorize this. The key insight: BM25 rewards documents where rare query terms appear frequently, but penalizes documents that are much longer than average (preventing long docs from always winning).
After running both BM25 and semantic search, you have two ranked lists. RRF merges them into one by assigning each document a score based on its rank in each list — not its raw score. This is robust because it is score-scale-agnostic: it doesn't matter that BM25 scores are in the hundreds and cosine similarities are 0–1.
def reciprocal_rank_fusion(
ranked_lists: list[list[str]],
k: int = 60,
) -> list[tuple[str, float]]:
"""
Merge multiple ranked lists using RRF.
k=60 is the standard constant that prevents very high ranks from dominating.
"""
scores: dict[str, float] = {}
for ranked_list in ranked_lists:
for rank, doc_id in enumerate(ranked_list, start=1):
scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
return sorted(scores.items(), key=lambda x: x[1], reverse=True)
# Example:
semantic_ranking = ["doc3", "doc1", "doc5", "doc2", "doc4"]
bm25_ranking = ["doc1", "doc3", "doc2", "doc4", "doc5"]
merged = reciprocal_rank_fusion([semantic_ranking, bm25_ranking])
print(merged)
# → [('doc3', 0.0317), ('doc1', 0.0317), ('doc2', 0.0226), ...]
# doc3 and doc1 both ranked highly in both lists → they winfrom rank_bm25 import BM25Okapi # pip install rank-bm25
import chromadb
from openai import OpenAI
import numpy as np
# Setup
openai = OpenAI()
chroma = chromadb.PersistentClient(path="./db")
collection = chroma.get_collection("documents")
# Pre-tokenize corpus for BM25
all_docs = collection.get(include=["documents", "metadatas"])
corpus_texts = all_docs["documents"]
corpus_ids = all_docs["ids"]
tokenized_corpus = [doc.lower().split() for doc in corpus_texts]
bm25 = BM25Okapi(tokenized_corpus)
def hybrid_search(query: str, n_results: int = 5) -> list[dict]:
# ── 1. BM25 keyword ranking ──────────────────────
tokenized_query = query.lower().split()
bm25_scores = bm25.get_scores(tokenized_query)
bm25_ranking = [corpus_ids[i] for i in np.argsort(bm25_scores)[::-1][:20]]
# ── 2. Semantic vector ranking ───────────────────
query_vec = openai.embeddings.create(
model="text-embedding-3-small", input=query
).data[0].embedding
semantic_results = collection.query(
query_embeddings=[query_vec], n_results=20,
)
semantic_ranking = semantic_results["ids"][0]
# ── 3. Fuse rankings with RRF ────────────────────
merged = reciprocal_rank_fusion([semantic_ranking, bm25_ranking])
# ── 4. Hydrate top-n results ─────────────────────
top_ids = [doc_id for doc_id, _ in merged[:n_results]]
hydrated = collection.get(ids=top_ids, include=["documents", "metadatas"])
return [
{"id": id_, "text": doc, "meta": meta}
for id_, doc, meta in zip(
hydrated["ids"], hydrated["documents"], hydrated["metadatas"]
)
]
results = hybrid_search("CVE-2024-1234 vulnerability details")
for r in results:
print(r["text"][:80])Conceptual queries, synonymous phrasing, questions and answers in different words
Exact phrase matching, product codes, proper nouns, very short corpora (<1K docs)
General-purpose production RAG — consistently the best default across query types
Some systems expose an alpha parameter instead of RRF:score = alpha × semantic_score + (1 - alpha) × bm25_score. The right alpha depends on your query mix:
alpha = 0.0 → pure keyword (BM25) alpha = 0.5 → balanced hybrid (good default) alpha = 1.0 → pure semantic # Rule of thumb: if users frequently search by exact name/code, lower alpha # If users ask open-ended conceptual questions, raise alpha
Next: even with perfect retrieval, you may be returning 20 chunks when only 3 are truly relevant. Re-ranking uses a cross-encoder model to re-score and re-order your retrieved candidates with much higher precision.