Project 42
Start learning
Module 2 of 5 · 50 min

Hybrid Search & Cross-Encoder Reranking

Combine sparse BM25 keyword search with dense embedding vectors and apply cross-encoder neural rerankers for optimal relevance.

Core concept

By the end

You will be able to

  • Architect reciprocal rank fusion (RRF) combining sparse BM25 and dense vector search.
  • Explain why cross-encoders achieve higher ranking precision than bi-encoder cosine similarity.
  • Deploy cross-encoder rerankers (Cohere Rerank, BGE-Reranker) in latency-bounded pipelines.
  • Benchmark NDCG@10 and Mean Reciprocal Rank (MRR) improvements across technical vocabularies.
01

Reciprocal Rank Fusion (RRF)

Dense vector embeddings excel at semantic paraphrasing but fail on exact part numbers, product IDs, and domain-specific acronyms. Sparse BM25 algorithms excel at exact keyword matches but fail on conceptual synonyms.

Reciprocal Rank Fusion combines both ranking lists into a single normalized score without requiring arbitrary score calibration or manual weight tuning.

Reciprocal Rank Fusion (RRF) Implementation
python
def reciprocal_rank_fusion(dense_results: list[str], sparse_results: list[str], k: int = 60) -> list[tuple[str, float]]:
    scores = {}
    for rank, doc in enumerate(dense_results):
        scores[doc] = scores.get(doc, 0.0) + 1.0 / (k + rank + 1)
    for rank, doc in enumerate(sparse_results):
        scores[doc] = scores.get(doc, 0.0) + 1.0 / (k + rank + 1)
    return sorted(scores.items(), key=lambda item: item[1], reverse=True)
02

Two-Stage Retrieval with Cross-Encoders

Bi-encoders generate document vectors independently, enabling fast index lookups at the cost of ignoring query-document cross-attention.

Cross-encoders jointly feed the query and candidate chunk through all attention layers, capturing intricate semantic nuances and eliminating false positives before prompt assembly.

Two-Stage Pipeline Architecture
text
User Query -> [Stage 1: Sparse (BM25) + Dense (ANN)] -> Top 50 Candidates -> [Stage 2: Cross-Encoder Reranker] -> Top 5 Gold Context Chunks -> LLM Generation

Practice activity

Benchmark Hybrid Search Against Dense-Only Search

  1. Construct a benchmark dataset with 20 technical queries containing exact product SKU numbers.
  2. Execute dense-only retrieval and measure Recall@5.
  3. Apply BM25 + Dense RRF fusion and BGE reranking, then measure the improvement in MRR.

What to produce

  • Benchmark test script and CSV results comparing Dense vs Hybrid vs Reranked MRR.

Reflect before continuing

Under what latency budget is a second-stage cross-encoder justifiable in high-concurrency production systems?

Evidence

Sources and verification

Knowledge check

Make it stick.

Pass at 80%

Choose the strongest answer for each question. Your attempts become part of your account transcript.

01Why is a cross-encoder more accurate than bi-encoder cosine similarity for reranking?