mediumAI Engineering

How do you manage conversations that outgrow the context window?

Q: How do you manage conversations that outgrow the context window?

**TL;DR.** Four patterns, usually combined: **sliding window** (keep recent turns), **running summary** (compress the old), **external memory** (store durable facts, retrieve on demand), and for agents, **state checkpoints** so work resumes in a fresh context. Keep IDs, constraints and decisions **verbatim** — summaries drift. turns 1-40 (old) turns 41-48 (recent) <rect class='d-box-accent' x='222' y='30' width='110' height='

771 views

Understand the problem

Sliding windows, summarization, and externalized memory for long-running chats and agents.

memorysummarizationcontext-windowchat

Attempt it yourself

Sketch your approach before reading the solution — that's what interviews test.

Stuck? AI Nudge Available

Get a conceptual hint to guide your logic without spoiling the final implementation.

Study the solution

The solution is waiting

Give it an honest attempt first — then compare your thinking with the full walkthrough.

Read the code

Summary + pins + recent turns

async function prepareHistory(turns: Turn[], pins: string[]) {
  const recent = turns.slice(-8);
  const evicted = turns.slice(0, -8);
  const summary = evicted.length
    ? await summarize(prevSummary, evicted)     // incremental, ~300 tokens
    : null;
  return [
    summary && { role: "user", content: "[Conversation so far]\n" + summary },
    pins.length && { role: "user", content: "[Pinned facts]\n- " + pins.join("\n- ") },
    ...recent,
  ].filter(Boolean);
}

Join the discussion

Discussion (0)

No responses yet. Be the first to share what you think.

How do temperature, top-p and other sampling parameters affect output?