Hybrid Search: Combining Sparse and Dense Retrievers with Reciprocal Rank Fusion

Retrieval systems form the foundation of search engines and RAG pipelines. Traditionally, search relied on sparse, keyword-based algorithms like TF-IDF and BM25. In recent years, dense retrieval using vector embeddings has gained immense popularity due to its ability to capture semantic meaning and synonyms.

However, neither approach is perfect. Dense search struggles with exact keyword matching, out-of-vocabulary terms, product serial numbers, and short queries. Sparse search excels at exact matches but fails completely when users search using synonyms or different phrasings.

Hybrid search solves these issues by executing sparse and dense retrieval in parallel and fusing their results.

Hybrid Search & Sparse-Dense Fusion

The Math Behind BM25 and Dense Retrieval

Sparse retrieval typically uses Okapi BM25, which ranks documents based on the query terms appearing in each document. The score for a document D given query Q containing terms q_i is:

\text{score}(D, Q) = \sum_{i=1}^{n} \text{IDF}(q_i) \cdot \frac{f(q_i, D) \cdot (k_1 + 1)}{f(q_i, D) + k_1 \cdot \left(1 - b + b \cdot \frac{|D|}{\text{avgdl}}\right)}

where f(q_i, D) is the term frequency of q_i in D, |D| is the document length, avgdl is the average document length in the corpus, and k_1 and b are parameters.

Dense retrieval computes the cosine similarity or dot product between the query vector E(q) and the document vector E(D):

\text{score}_{dense}(q, D) = E(q) \cdot E(D)

Because sparse and dense scores occupy entirely different mathematical distributions, they cannot be summed or compared directly. To combine them, we use Reciprocal Rank Fusion (RRF).

Reciprocal Rank Fusion (RRF)

RRF is a simple, parameter-free algorithm that scores documents based on their rank positions in both the sparse and dense result sets. The RRF score for document d is:

RRF(d) = \sum_{m \in M} \frac{1}{k + r_m(d)}

where M is the set of retrievers (sparse and dense), r_m(d) is the rank of document d in retriever m, and k is a smoothing constant (typically set to 60).

Python Implementation of RRF

Let's implement a clean RRF function to merge sparse and dense retrieval lists:

def reciprocal_rank_fusion(sparse_results, dense_results, k=60, top_n=10):
    # Fuses sparse and dense results using RRF.
    # sparse_results and dense_results are lists of document IDs in ranked order.
    rrf_scores = {}

    # Helper function to add scores
    def add_scores(results):
        for rank, doc_id in enumerate(results, start=1):
            if doc_id not in rrf_scores:
                rrf_scores[doc_id] = 0.0
            rrf_scores[doc_id] += 1.0 / (k + rank)

    add_scores(sparse_results)
    add_scores(dense_results)

    # Sort documents by their RRF score
    sorted_docs = sorted(rrf_scores.items(), key=lambda item: item[1], reverse=True)
    return sorted_docs[:top_n]

# Example run
sparse = ["doc1", "doc3", "doc5", "doc7"]
dense = ["doc3", "doc2", "doc1", "doc8"]
fused = reciprocal_rank_fusion(sparse, dense, k=60, top_n=5)
print("Fused RRF results:", fused)

Implementing Hybrid Search in Production Vector DBs

Most modern vector databases (like Qdrant, Pinecone, and Weaviate) support hybrid search natively.

• Qdrant allows you to configure dense vectors alongside sparse vectors (using algorithms like SPLADE or BM25 indexers) and perform a single hybrid query specifying an RRF fusion key.
• Pinecone supports sparse-dense vectors under its hybrid index mode.
• Weaviate has built-in hybrid query parameters that merge BM25 and vector scores using configurable weights or RRF.

Conclusion

Hybrid search represents the state-of-the-art for retrieval systems. By combining the lexical precision of sparse search with the semantic recall of dense embeddings via Reciprocal Rank Fusion, it delivers robust performance across diverse, real-world user queries.