hardAI Engineering

What are common RAG failure modes and how do you debug them?

Q: What are common RAG failure modes and how do you debug them?

**TL;DR.** The big four: **answer never retrieved** (vocabulary mismatch, bad chunking, wrong k), **retrieved but ignored** (buried mid-context, conflicting chunks), **stale/duplicate index**, and **multi-hop questions** top-k cannot serve. Debug by **logging the retrieved set per query** and triaging: gold chunk absent → retrieval problem; present but unused → generation problem. gold chunk in top-k? no <text class='d-sub' x

562 views

Understand the problem

Retrieval misses, lost-in-the-middle, stale indexes, chunk fragmentation — a triage guide.

ragdebuggingfailure-modesretrieval

Attempt it yourself

Sketch your approach before reading the solution — that's what interviews test.

Stuck? AI Nudge Available

Get a conceptual hint to guide your logic without spoiling the final implementation.

Study the solution

The solution is waiting

Give it an honest attempt first — then compare your thinking with the full walkthrough.

Read the code

Per-request retrieval logging for triage

const chunks = await retrieve(query, { k: 8 });
const reply = await generate(query, chunks);

await log.rag({
  query,
  retrieved: chunks.map((c) => ({ id: c.id, score: c.score, doc: c.docId })),
  answer: reply.text,
  feedback: null,               // filled by thumbs up/down later
});
// weekly: cluster thumbs-down rows by "gold chunk retrieved?" → fix the bigger pile

Join the discussion

Discussion (0)

No responses yet. Be the first to share what you think.

How do you version, test and safely roll out prompt changes?

What is prompt injection and how do you defend against it?