loading…

Context Rot

Context Rot is the phenomenon where an LLM’s reasoning accuracy, instruction adherence, and recall degrade as the context window fills with intermediate conversational turns, noisy tool outputs, and historical artifacts.

The Mechanism

Transformer attention is not uniform across thousands of tokens:

1. Lost in the Middle: Information placed in the center of a long context is retrieved with significantly lower fidelity than tokens at the very beginning (system prompt) or the very end (latest turn).
2. Attention Dilution: Every irrelevant token in context consumes an attention budget, increasing the probability of hallucinations and missed edge cases.
3. Compounding Noise: When an agent misinterprets a noisy tool result and replies with flawed assumptions, that flawed exchange remains in context, poisoning subsequent reasoning steps.

Mitigations

• Context Clearing (/clear): Reset the context completely between distinct, modular tasks.
• Handoff Artifacts: Compress multi-turn discussions into clean, structured handoff documents rather than carrying raw conversation logs forward.
• Subagent Delegation: Offload verbose investigations (searches, log reading, scraping) to isolated subagents and return only the distilled summary.
• Strict Progressive Disclosure: Load references and documentation on-demand rather than dumping everything into the initial system prompt.

Rule of Thumb

Compaction triggers on token budget pressure, not on quality degradation you can observe - by the time answers degrade, the rot predates them.

flowchart LR
    A[Long session] --> B[Turns + tool outputs accumulate]
    B --> C[Attention diluted]
    C --> D[Instruction adherence drops]
    C --> E[Recall of early context drops]
    D & E --> F[Fix: compact / restart / re-inject constraints]

Failure Modes

Symptom	Root cause	Fix
Answers degrade on long tasks	Attention diluted by accumulated turns	Compact or restart; never let turns pile unbounded
Early instructions forgotten	Lost-in-the-middle placement	Re-state critical constraints at the END of context
Tool outputs flood the window	Verbose results kept verbatim	Summarize tool output before it enters history

Related

smart zone, handoff artifacts, context engineering, prompt bloat, subagents and context management, ai coding taxonomy.