easyAI Engineering

What is a context window and what happens when you exceed it?

Q: What is a context window and what happens when you exceed it?

**TL;DR.** The context window is the **maximum tokens (input + output)** a model can attend to per request. Exceed it and you get an **API error or truncation**; apps cope by **trimming, summarizing, or retrieving** only what matters. Long contexts also degrade recall (**lost in the middle**) and cost more. system history retrieved docs</

172 views

Understand the problem

The model's working memory: what fits, what falls out, and how apps deal with the limit.

context-windowlimitsfundamentals

Attempt it yourself

Sketch your approach before reading the solution — that's what interviews test.

Stuck? AI Nudge Available

Get a conceptual hint to guide your logic without spoiling the final implementation.

Study the solution

The solution is waiting

Give it an honest attempt first — then compare your thinking with the full walkthrough.

Read the code

Guarding the window before a chat call

const WINDOW = 200_000;
const RESERVED_OUTPUT = 4_096;

function fitMessages(system: string, turns: Msg[], docs: Doc[]) {
  let budget = WINDOW - RESERVED_OUTPUT - countTokens(system);
  const keptDocs  = takeWhileUnderBudget(rankByRelevance(docs), 0.4 * budget);
  const keptTurns = takeRecentUnderBudget(turns, budget - tokensOf(keptDocs));
  return { system, docs: keptDocs, turns: keptTurns };
}

Join the discussion

Discussion (0)

No responses yet. Be the first to share what you think.

What is the difference between zero-shot, one-shot and few-shot prompting?

How do you choose the right model for a task?