Skip to content
Call wiki

How much context can you feed an AI before quality drops, and when should you restart?

Last updated 2026-08-17 · Confidence: documented — Anthropic’s docs and engineering blog plus Chroma’s measured study; the specific token thresholds heard on calls are rules of thumb, not documented anywhere.

There is no safe token number: degradation is gradual, task-dependent, and starts well below the window limit — so restart on behavior (quality dropping, task switching), not on a count.

Degradation is real, gradual, and measured

Section titled “Degradation is real, gradual, and measured”

Anthropic’s context-windows doc states it plainly: “more context isn’t automatically better. As token count grows, accuracy and recall degrade, a phenomenon known as context rot.” Their engineering post frames it as an “attention budget” every added token depletes.

Chroma’s study (July 2025, 18 models incl. Claude 4, GPT-4.1, Gemini 2.5) measured it: performance varies with input length “even on simple tasks,” and similar-but-irrelevant material (distractors) steepens the drop. Lost in the Middle (2023) adds position: models use context best at the beginning and end, worst in the middle.

Both directions fail — curate, don’t just cut

Section titled “Both directions fail — curate, don’t just cut”

For chat users the commoner failure is still missing context — a review that quietly read only the summaries of the meeting transcripts it was given. “More context is better than less” holds while the added tokens are relevant; the research above is about volume that isn’t. The target is all-relevant, minimal-noise.

When to restart

Section titled “When to restart”

No vendor publishes a restart threshold; “delegate past ~200k of a 1M window” is a call rule of thumb of roughly the right shape (current Claude models default to 1M tokens; Sonnet 4.5 and earlier get 200k). The documented triggers are behavioral: compact “when context starts affecting performance or before a long new task,” and clear between tasks because “old conversation crowds out the files you need next.” A degrading chat also keeps re-reading its own earlier outputs — they count toward the window like input — which is why stale sessions lock into patterns a fresh one escapes.

A usable rule: for a bounded task, feed everything relevant — tens of documents is fine. Near a hundred, have subagents read and return short summaries (Anthropic’s pattern: 1,000–2,000 tokens each). Restart at the first sign of pattern-lock or a task switch, carrying a written progress summary — standing instructions reload automatically.

Sources

Section titled “Sources”