mediumAI Engineering

What is reranking and why add it after retrieval?

Q: What is reranking and why add it after retrieval?

**TL;DR.** A **cross-encoder** reranker scores each (query, document) pair **jointly** — far more accurate than embedding comparison, far too slow for the whole corpus. So: fast retrieval pulls top-50/100 (**recall**), the reranker reorders and you keep 5-10 (**precision**). Usually the biggest retrieval upgrade per line of code. corpus millions of chunks ANN <text class='d-text' x=

442 views

Understand the problem

Cross-encoders as a precision stage: cheap recall first, expensive relevance second.

rerankingcross-encoderretrievalrag

Attempt it yourself

Sketch your approach before reading the solution — that's what interviews test.

Stuck? AI Nudge Available

Get a conceptual hint to guide your logic without spoiling the final implementation.

Study the solution

The solution is waiting

Give it an honest attempt first — then compare your thinking with the full walkthrough.

Read the code

Two-stage retrieval with a score floor

const candidates = await hybridSearch(query, { limit: 80 });

const scored = await reranker.rank({
  query,
  documents: candidates.map((c) => c.text),
  topN: 8,
});

const kept = scored.filter((r) => r.relevance > 0.35);   // calibrated floor
if (kept.length === 0) return abstain();                 // no junk context
return kept.map((r) => candidates[r.index]);

Join the discussion

Discussion (0)

No responses yet. Be the first to share what you think.

How do you reduce hallucinations in a production LLM application?

How do you design good tool definitions for an LLM?