Hybrid search: combining BM25 and vector similarity for better RAG

The retrieval gap in single-method search

Every RAG system lives or dies by its retrieval step. If the documents retrieved from your corpus do not contain the information needed to answer the question, the LLM cannot synthesise a correct answer regardless of how capable it is. The quality ceiling of a RAG system is set by retrieval precision and recall.

Two retrieval paradigms dominate production systems today. Sparse keyword search, most commonly implemented as BM25, scores documents by term frequency weighted against inverse document frequency. It excels at exact matches: a query for "RFC 2822" will find documents containing that exact string reliably. Vector similarity search encodes both the query and documents into dense embedding vectors and ranks by cosine or dot-product similarity. It excels at semantic matches: a query for "email format standard" will find documents about RFC 2822 even if those words do not appear in the document title.

Neither approach dominates the other across all query types. BM25 fails on paraphrase queries and domain-specific terminology that is semantically related but lexically different. Vector search fails on exact-match queries—product codes, error codes, proper nouns, acronyms—because these rare tokens may not cluster meaningfully in embedding space, especially if the embedding model was not trained on your domain.

Hybrid search fuses both signals. The empirical evidence across BEIR and MTEB benchmarks consistently shows that hybrid retrieval outperforms either method alone by 5–20% on nDCG@10, with the gap widening on domain-specific corpora.

BM25 fundamentals

BM25 (Best Match 25) is a probabilistic ranking function that improves on TF-IDF by accounting for document length normalisation and term frequency saturation. The score for document $d$ given query $q$ is:

score(d, q) = Σ IDF(t) * (f(t,d) * (k1 + 1)) / (f(t,d) + k1 * (1 - b + b * |d| / avgdl))

Where f(t,d) is the term frequency of term t in document d, |d| is document length, avgdl is average document length across the corpus, and k1 and b are tunable parameters (typical values: k1=1.5, b=0.75).

The IDF component down-weights common terms across the corpus and up-weights rare, discriminative terms. This is why BM25 is particularly effective for technical content with domain-specific jargon: a term like "segfault" appearing in a query is highly informative in a programming documentation corpus.

In Python, the rank_bm25 library provides a clean implementation:

from rank_bm25 import BM25Okapi
import re

def tokenize(text: str) -> list[str]:
    # Lowercase, split on non-alphanumeric, filter empties
    return [token for token in re.split(r"[^a-z0-9]+", text.lower()) if token]

class BM25Index:
    def __init__(self, documents: list[str]):
        self.documents = documents
        tokenized = [tokenize(doc) for doc in documents]
        self.bm25 = BM25Okapi(tokenized, k1=1.5, b=0.75)

    def search(self, query: str, top_k: int = 20) -> list[tuple[int, float]]:
        query_tokens = tokenize(query)
        scores = self.bm25.get_scores(query_tokens)
        # Return (index, score) pairs sorted descending
        ranked = sorted(enumerate(scores), key=lambda x: x[1], reverse=True)
        return ranked[:top_k]

For production scale (millions of documents), use Elasticsearch or OpenSearch instead of an in-memory implementation. Both expose BM25 natively and allow field-level boosting, which is useful for weighting title matches more heavily than body matches.

Vector similarity search

Dense retrieval encodes documents into fixed-dimensional vectors using an embedding model, then at query time encodes the query into the same space and retrieves the nearest neighbours.

import numpy as np
from anthropic import Anthropic

client = Anthropic()

def embed_texts(texts: list[str], model: str = "voyage-3") -> np.ndarray:
    """Get embeddings using Voyage AI via the Anthropic client."""
    # In practice use the voyage-ai SDK or OpenAI embeddings SDK
    # This illustrates the interface pattern
    import voyageai
    vo = voyageai.Client()
    result = vo.embed(texts, model=model, input_type="document")
    return np.array(result.embeddings)

class VectorIndex:
    def __init__(self, documents: list[str]):
        self.documents = documents
        self.embeddings = embed_texts(documents)
        # Normalise for cosine similarity via dot product
        norms = np.linalg.norm(self.embeddings, axis=1, keepdims=True)
        self.embeddings = self.embeddings / np.maximum(norms, 1e-8)

    def search(self, query: str, top_k: int = 20) -> list[tuple[int, float]]:
        query_embedding = embed_texts([query], input_type="query")[0]
        query_norm = np.linalg.norm(query_embedding)
        query_embedding = query_embedding / max(query_norm, 1e-8)

        scores = self.embeddings @ query_embedding
        ranked_indices = np.argsort(scores)[::-1][:top_k]
        return [(int(idx), float(scores[idx])) for idx in ranked_indices]

For production deployments, use a dedicated vector database: pgvector (PostgreSQL extension), Pinecone, Weaviate, or Qdrant. These support approximate nearest neighbour (ANN) search via HNSW or IVF indexes, which make retrieval over millions of vectors sub-second.

-- pgvector setup
CREATE EXTENSION vector;

CREATE TABLE documents (
  id BIGSERIAL PRIMARY KEY,
  content TEXT NOT NULL,
  embedding vector(1024),  -- dimension depends on embedding model
  metadata JSONB
);

CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

-- Semantic search query
SELECT id, content, 1 - (embedding <=> $1::vector) AS similarity
FROM documents
ORDER BY embedding <=> $1::vector
LIMIT 20;

Reciprocal Rank Fusion: fusing the two result sets

The central challenge of hybrid search is combining scores from two different systems that operate on entirely different scales. A BM25 score of 12.4 and a cosine similarity of 0.87 cannot be directly added—they have different distributions, ranges, and semantics.

Reciprocal Rank Fusion (RRF) sidesteps this problem elegantly by operating on ranks rather than scores. The fused score for document $d$ is:

RRF(d) = Σ_r 1 / (k + rank_r(d))

Where the sum is over each retrieval system $r$, rank_r(d) is the rank of document $d$ in system $r$'s results (1-indexed), and k is a smoothing constant (typically 60) that prevents extreme score compression for top-ranked documents.

def reciprocal_rank_fusion(
    result_lists: list[list[tuple[int, float]]],
    k: int = 60,
    top_k: int = 10,
) -> list[tuple[int, float]]:
    """
    Fuse multiple ranked result lists using RRF.
    Each result list is a list of (document_id, score) tuples sorted descending.
    Returns fused (document_id, rrf_score) tuples sorted descending.
    """
    rrf_scores: dict[int, float] = {}

    for result_list in result_lists:
        for rank, (doc_id, _score) in enumerate(result_list, start=1):
            rrf_scores[doc_id] = rrf_scores.get(doc_id, 0.0) + 1.0 / (k + rank)

    fused = sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True)
    return fused[:top_k]


class HybridSearchIndex:
    def __init__(self, documents: list[str]):
        self.bm25_index = BM25Index(documents)
        self.vector_index = VectorIndex(documents)

    def search(self, query: str, top_k: int = 10) -> list[tuple[int, float]]:
        bm25_results = self.bm25_index.search(query, top_k=50)
        vector_results = self.vector_index.search(query, top_k=50)

        fused = reciprocal_rank_fusion(
            [bm25_results, vector_results],
            k=60,
            top_k=top_k,
        )
        return fused

RRF is robust and works well out of the box. Its main limitation is that it treats both retrieval systems with equal weight. When you have strong evidence that one method consistently outperforms the other for your query distribution, weighted RRF (multiplying each system's contribution by a weight) or a learned reranker will do better.

Alternative fusion: weighted linear combination with score normalisation

When you have labelled query-document relevance data, you can tune the weights of a linear combination directly. This requires normalising BM25 and vector scores to a common range first.

def min_max_normalise(scores: list[tuple[int, float]]) -> list[tuple[int, float]]:
    if not scores:
        return scores
    values = [s for _, s in scores]
    min_val, max_val = min(values), max(values)
    if max_val == min_val:
        return [(doc_id, 1.0) for doc_id, _ in scores]
    return [
        (doc_id, (score - min_val) / (max_val - min_val))
        for doc_id, score in scores
    ]

def weighted_hybrid_search(
    bm25_results: list[tuple[int, float]],
    vector_results: list[tuple[int, float]],
    alpha: float = 0.5,  # weight for vector; (1-alpha) for BM25
    top_k: int = 10,
) -> list[tuple[int, float]]:
    bm25_norm = dict(min_max_normalise(bm25_results))
    vector_norm = dict(min_max_normalise(vector_results))

    all_ids = set(bm25_norm) | set(vector_norm)
    combined: dict[int, float] = {}
    for doc_id in all_ids:
        bm25_score = bm25_norm.get(doc_id, 0.0)
        vec_score = vector_norm.get(doc_id, 0.0)
        combined[doc_id] = (1 - alpha) * bm25_score + alpha * vec_score

    sorted_results = sorted(combined.items(), key=lambda x: x[1], reverse=True)
    return sorted_results[:top_k]

The alpha parameter is your primary tuning lever. Values closer to 1.0 favour semantic search; values closer to 0.0 favour keyword search. For query-heavy, exact-match workloads (product search, technical documentation), start at 0.3. For open-ended semantic queries (question answering over prose), start at 0.7. Run offline evaluations with your labelled dataset to find the optimal value.

Implementing hybrid search with Elasticsearch

Elasticsearch supports hybrid search natively via the knn query combined with traditional BM25 queries using the bool combinator:

{
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "content": {
              "query": "{{user_query}}",
              "boost": 0.5
            }
          }
        }
      ]
    }
  },
  "knn": {
    "field": "embedding",
    "query_vector": [/* your query vector */],
    "k": 50,
    "num_candidates": 100,
    "boost": 0.5
  },
  "size": 10
}

The boost values here are your alpha parameter—they control the relative weight of BM25 versus vector scores. Elasticsearch normalises each to [0, 1] before applying the boost, so you can reason about them on the same scale.

Adding a cross-encoder reranker

RRF fuses results from two bi-encoders (BM25 is a bag-of-words bi-encoder; vector similarity uses a dense bi-encoder). A cross-encoder can significantly improve final precision by jointly encoding the query and each candidate document, allowing the model to reason about their relationship rather than computing scores independently.

Cross-encoders are expensive—they must be run once per (query, candidate) pair—so they are applied only to the top-K candidates from the hybrid retrieval stage, not the full corpus.

from sentence_transformers import CrossEncoder

reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")

def rerank(
    query: str,
    candidates: list[tuple[int, str]],  # (doc_id, content)
    top_k: int = 5,
) -> list[tuple[int, float]]:
    pairs = [(query, content) for _, content in candidates]
    scores = reranker.predict(pairs)
    ranked = sorted(
        zip([doc_id for doc_id, _ in candidates], scores),
        key=lambda x: x[1],
        reverse=True,
    )
    return ranked[:top_k]

A common production pattern is: hybrid retrieval (top 50) → cross-encoder reranker (top 5) → LLM generation. The two-stage pipeline keeps latency manageable while capturing the precision of cross-encoder scoring.

Measuring retrieval quality

Avoid tuning hybrid search parameters without measurement. The relevant metrics are:

• Recall@K: what fraction of relevant documents appear in the top K results?
• Precision@K: what fraction of the top K results are relevant?
• nDCG@K: normalised discounted cumulative gain, which accounts for rank position (relevant documents ranked higher contribute more).
• MRR: mean reciprocal rank, the average of 1/rank for the first relevant document.

To compute these, you need a labelled evaluation set: a collection of queries paired with their ground-truth relevant documents. Building this set is the hardest part of improving retrieval. Options include:

1. Manual labelling: domain experts label query-document pairs. High quality, expensive.
2. LLM-generated labels: use an LLM to judge relevance for a sample of retrieved documents. Faster, somewhat noisy.
3. Click-through data: if your system is live, click and engagement signals proxy for relevance. Requires sufficient traffic.

def compute_recall_at_k(
    retrieved_ids: list[int],
    relevant_ids: set[int],
    k: int,
) -> float:
    retrieved_top_k = set(retrieved_ids[:k])
    if not relevant_ids:
        return 0.0
    return len(retrieved_top_k & relevant_ids) / len(relevant_ids)

def compute_ndcg_at_k(
    retrieved_ids: list[int],
    relevant_ids: set[int],
    k: int,
) -> float:
    import math
    dcg = sum(
        1.0 / math.log2(rank + 2)
        for rank, doc_id in enumerate(retrieved_ids[:k])
        if doc_id in relevant_ids
    )
    ideal_hits = min(len(relevant_ids), k)
    idcg = sum(1.0 / math.log2(rank + 2) for rank in range(ideal_hits))
    return dcg / idcg if idcg > 0 else 0.0

Run these metrics separately for BM25-only, vector-only, and hybrid retrieval on your evaluation set. The comparison will give you a clear picture of where each method wins and guide your alpha tuning.

Production architecture considerations

Several practical concerns arise when moving hybrid search from prototype to production.

Index synchronisation: BM25 and vector indexes must stay in sync as documents are added, updated, or deleted. Build a document ingestion pipeline that writes to both indexes atomically, or accept eventual consistency with a reconciliation job.

Embedding model updates: when you upgrade the embedding model, all existing document embeddings become incompatible with new query embeddings. Plan for a re-indexing strategy—either full corpus re-embedding (expensive but clean) or maintaining two embedding columns during migration.

Query latency budget: a round-trip to BM25 + a round-trip to the vector index + RRF fusion + an optional reranker pass adds latency. Measure end-to-end P50, P95, and P99. If you have strict latency SLAs, run the two retrieval calls in parallel and consider skipping the cross-encoder for queries that already return high-confidence results.

Sparse-dense alignment: some vector databases (Weaviate, Qdrant) support hybrid search natively, running BM25-equivalent sparse retrieval and dense retrieval in a single engine. This eliminates the need to maintain two separate systems and simplifies the operational burden significantly.

Hybrid search is not a silver bullet—its advantage over pure vector search shrinks as embedding model quality improves and as training corpora cover your domain more thoroughly. But for most production RAG systems today, particularly those in specialised domains with significant exact-match query patterns, hybrid retrieval is the most reliable way to push recall and precision higher than either method achieves alone.