note · growing
Hybrid search and reranking
Dense + BM25, fused with RRF, cut to top-5 by a cross-encoder. Retrieve wide, hand over narrow.
Dense vectors capture meaning and miss exact matches: error codes, SKUs, person names, version numbers. BM25 is good at exactly those.
Densetop-50
BM25top-50
RRFfuse
Rerankcross-encoder
Top-5
Why this order
- Retrieve wide, hand over narrow. Fetching top-50 and cutting to top-5 is both cheaper and better than dumping a raw top-20 into the context.
- RRF fuses two lists without any attempt to normalise their scores; it adds up ranks instead.
- A cross-encoder reads the question and the chunk together. It catches the relationship a bi-encoder misses — at the cost of latency, which is why it comes last and over few candidates.
Ask for chunk ids in the answer. A RAG system that cites nothing does not remove hallucination — it just makes it invisible.