Hybrid Search: Combining Sparse and Dense Retrievers with Reciprocal Rank Fusion

Retrieval systems form the foundation of search engines and RAG pipelines. Traditionally, search relied on sparse, keyword-based algorithms like TF-IDF and BM25. In recent years, dense retrieval using vector embeddings has gained immense popularity due to its ability to capture semantic meaning and synonyms.

However, neither approach is perfect. Dense search struggles with exact keyword matching, out-of-vocabulary terms, product serial numbers, and short queries. Sparse search excels at exact matches but fails completely when users search using synonyms or different phrasings.

Hybrid search solves these issues by executing sparse and dense retrieval in parallel and fusing their results.

Hybrid Search & Sparse-Dense Fusion

The Math Behind BM25 and Dense Retrieval

Sparse retrieval typically uses Okapi BM25, which ranks documents based on the query terms appearing in each document. The score for a document D given query Q containing terms q_i is:

\text{score}(D, Q) = \sum_{i=1}^{n} \text{IDF}(q_i) \cdot \frac{f(q_i, D) \cdot (k_1 + 1)}{f(q_i, D) + k_1 \cdot \left(1 - b + b \cdot \frac{|D|}{\text{avgdl}}\right)}

where f(q_i, D) is the term frequency of q_i in D, |D| is the document length, avgdl is the average document length in the corpus, and k_1 and b are parameters.

Dense retrieval computes the cosine similarity or dot product between the query vector E(q) and the document vector E(D):

\text{score}_{dense}(q, D) = E(q) \cdot E(D)

Because sparse and dense scores occupy entirely different mathematical distributions, they cannot be summed or compared directly. To combine them, we use Reciprocal Rank Fusion (RRF).

Reciprocal Rank Fusion (RRF)

RRF is a simple, parameter-free algorithm that scores documents based on their rank positions in both the sparse and dense result sets. The RRF score for document d is:

RRF(d) = \sum_{m \in M} \frac{1}{k + r_m(d)}

where M is the set of retrievers (sparse and dense), r_m(d) is the rank of document d in retriever m, and k is a smoothing constant (typically set to 60).

Python Implementation of RRF

Let's implement a clean RRF function to merge sparse and dense retrieval lists:

def reciprocal_rank_fusion(sparse_results, dense_results, k=60, top_n=10):
    # Fuses sparse and dense results using RRF.
    # sparse_results and dense_results are lists of document IDs in ranked order.
    rrf_scores = {}

    # Helper function to add scores
    def add_scores(results):
        for rank, doc_id in enumerate(results, start=1):
            if doc_id not in rrf_scores:
                rrf_scores[doc_id] = 0.0
            rrf_scores[doc_id] += 1.0 / (k + rank)

    add_scores(sparse_results)
    add_scores(dense_results)

    # Sort documents by their RRF score
    sorted_docs = sorted(rrf_scores.items(), key=lambda item: item[1], reverse=True)
    return sorted_docs[:top_n]

# Example run
sparse = ["doc1", "doc3", "doc5", "doc7"]
dense = ["doc3", "doc2", "doc1", "doc8"]
fused = reciprocal_rank_fusion(sparse, dense, k=60, top_n=5)
print("Fused RRF results:", fused)

Implementing Hybrid Search in Production Vector DBs

Most modern vector databases (like Qdrant, Pinecone, and Weaviate) support hybrid search natively.

• Qdrant allows you to configure dense vectors alongside sparse vectors (using algorithms like SPLADE or BM25 indexers) and perform a single hybrid query specifying an RRF fusion key.
• Pinecone supports sparse-dense vectors under its hybrid index mode.
• Weaviate has built-in hybrid query parameters that merge BM25 and vector scores using configurable weights or RRF.

Conclusion

RRF's great virtue is also its blind spot: because it uses only rank positions, it throws away score magnitude entirely. A document that BM25 matched with overwhelming lexical confidence and one that squeaked into first place on a weak match contribute identically to the fused result. Most of the time that robustness is what you want — it is exactly why RRF needs no tuning and no score normalization. But when one retriever is confidently right and the other is returning noise, RRF has no way to tell, and it will faithfully promote the noise into your top results. If you have a labeled query set and can measure it, weighted score fusion with normalized scores sometimes beats RRF by a meaningful margin. Without that evaluation set, stick with RRF; untuned weighted fusion is reliably worse than untuned RRF.

The k=60 constant deserves a note too, since it gets copied around as though it were derived. It comes from the original 2009 Cormack et al. paper, tuned on TREC collections that look nothing like your corpus. Larger k flattens the contribution of rank differences, making the fusion more democratic; smaller k sharpens the advantage of top-ranked documents. If you have evaluation data, it is one of the cheapest parameters to sweep.

Where hybrid search genuinely is not worth it: corpora where the vocabulary is uniform and the queries are natural language questions. Dense retrieval alone handles that well, and you are paying for a second index, a second query path, and the operational cost of keeping them in sync for marginal gain. The case for hybrid is strongest when your corpus contains identifiers that embeddings cannot represent — SKUs, error codes, function names, drug names, legal citations. An embedding model has no useful representation of ERR_CONN_5521, and no amount of semantic sophistication fixes that; BM25 finds it instantly. If your users search for strings rather than concepts, add sparse retrieval first and worry about fusion tuning second.