What is hybrid search in RAG?

Hybrid search runs keyword search, usually BM25, and dense vector search side by side, then combines the two ranked lists into one. The combination is usually reciprocal rank fusion or an alpha weighted score blend. It exists because dense search misses exact IDs, names, and codes, while BM25 misses paraphrase. On our test, dense search alone scored nDCG at 10 of 0.796 and BM25 alone 0.776, while fusion with k 60 scored 0.787, so hybrid is not automatically better than dense alone.

What is reciprocal rank fusion and which k value works best?

Reciprocal rank fusion scores each document as the sum over input lists of 1 divided by k plus its rank, ignoring raw scores entirely. The original 2009 paper fixed k at 60 from a pilot study and found varying k from 10 to 100 changed MAP by less than 0.003. On our test k 10 gave nDCG at 10 of 0.796 and k 60 gave 0.787, so either is defensible. Pick 60, the published default, and spend the tuning effort elsewhere.

What does a reranker do in RAG?

A reranker, usually a cross encoder, reads the query and each candidate document together and scores how well they match, then reorders the top candidates from the first retrieval stage. On our test reranking the top 20 with a MiniLM cross encoder raised nDCG at 10 from 0.796 to 0.820 and MRR from 0.744 to 0.775. It is precise and slow: 6.4 seconds per query on our CPU for 20 candidates.

Can reranking fix bad retrieval?

No. A reranker only reorders the candidates the first stage returned. In our test the question about BPE training on the Sennrich toy example was missed entirely by dense retrieval, so reranking could not rescue it, while BM25 had ranked it first. If the gold passage is not in the candidate set, no ranker can find it. Fix recall in the retriever before adding a reranker.

Is BM25 still useful when you have embeddings?

Yes, for exact tokens. BM25 alone scored recall at 10 of 0.88 on our test, below dense search at 0.96, but it ranked first on questions dense search missed, including one about a toy example with rare words and the classic ticket ID case. Keep BM25 in the pipeline for IDs, names, error codes, and version numbers, and fuse it with dense search rather than replacing it.

What is ColBERT and when does it fit a RAG pipeline?

ColBERTv2 is a late interaction retriever: query and passage are encoded separately into one vector per token, and relevance is MaxSim, the sum over query tokens of the best matching passage token. It scores 39.7 MRR at 10 on MS MARCO, up from 36.0 for the original ColBERT, and was best on 22 of 28 out of domain tests. Use it when pure dense search is too weak and full cross encoder reranking is too slow.

RAG Deep: Hybrid Search and Reranking, Measured