mediumAI Engineering

What chunking strategies exist and how does chunk size affect RAG quality?

Q: What chunking strategies exist and how does chunk size affect RAG quality?

**TL;DR.** Chunking decides what your retriever can ever return. Options: **fixed token windows** (+overlap), **structure-aware splits** (headings, paragraphs, code blocks), and **parent-child** (embed small, return big). Small chunks match precisely but fragment meaning; large chunks keep context but dilute embeddings. Start ~**300-800 tokens, 10-20% overlap**, respect structure — then tune with retrieval evals. small chunks <text cla

615 views

Understand the problem

Fixed windows, semantic splits, parent-child chunks — the retrieval decision made at index time.

chunkingragindexing

Attempt it yourself

Sketch your approach before reading the solution — that's what interviews test.

Stuck? AI Nudge Available

Get a conceptual hint to guide your logic without spoiling the final implementation.

Study the solution

The solution is waiting

Give it an honest attempt first — then compare your thinking with the full walkthrough.

Read the code

Structure-aware chunking with context header

def chunk_doc(doc):
    for section in split_by_headings(doc.markdown):        # structure first
        path = " > ".join(section.heading_path)            # "Refunds > EU"
        for piece in window(section.text, tokens=500, overlap=60):
            yield {
                "text": path + ": " + piece,               # self-describing chunk
                "doc_id": doc.id,
                "parent": section.id,                      # return this at answer time
            }

Join the discussion

Discussion (0)

No responses yet. Be the first to share what you think.

How do you design human-in-the-loop approval for agent actions?

What is prompt caching and how does it reduce cost and latency?