ARTHAVYA
← All posts
ragretrievalgenaievals

RAG That Doesn't Hallucinate (Much)

Practical retrieval patterns that reduce hallucination rates in real customer deployments—hybrid search, rerankers, and citation-first prompting.

Nikhil G.6 min read

Vector search alone doesn't cut it. Every serious RAG system we've shipped combines lexical search, semantic search, and a reranker — and then constrains the model with citation-first prompting.

#The stack

  • BM25 for exact-term recall (product SKUs, error codes, names).
  • Dense embeddings for semantic recall.
  • Reciprocal rank fusion to merge results.
  • Cross-encoder reranker on the top 30 to pick the final 5.
const results = await Promise.all([
  bm25.search(query, { k: 30 }),
  vectors.search(await embed(query), { k: 30 }),
]);

const fused = reciprocalRankFusion(results, { k: 60 });
const reranked = await reranker.rank(query, fused.slice(0, 30));
const context = reranked.slice(0, 5);

#Citation-first prompting

Instead of asking the model to answer and cite, ask it to cite and then answer. The order matters more than you'd think.

Given the following passages, first list which passages contain the answer by ID. Then write an answer using only those passages. If no passage contains the answer, say so.

This one change dropped fabricated citations in our test set from 11% to under 2%.

#Measuring what matters

Track answer groundedness (does every claim map to a retrieved passage?) not just answer quality. A "correct" answer that isn't grounded is a hallucination waiting to happen on the next question.