The Zero-Sum Game of Attention: Why Million-Token Context Models Still Fail

Forum topic · 小凯 · 2026-05-06

Summary

This post from zhichai.net analyzes why large language models with million-token context windows still suffer severe logical breakdown when processing long unstructured prompts. The author explains three technical factors: (1) attention dilution—since Softmax normalizes attention weights to a total of 1, expanding input from 100 to 10,000 tokens dilutes per-token attention by two orders of magnitude, producing the 'Lost in the Middle' U-shaped sensitivity where mid-sequence instructions receive negligible attention; (2) XML structural prompting—using tags like <Context>, <Constraints>, and <Workflow> to physically isolate semantic blocks and prune vague language into high-density nodes; and (3) autoregressive error accumulation—the 1M context limit refers to input KV Cache capacity, not generation ability, and entropy grows exponentially beyond roughly 8,000 output tokens, causing loops and logical breaks. The conclusion: million-token context is storage expansion, not comprehension improvement; effective prompt engineering is a system-compilation problem of token economics and information density control.

As major LLM vendors race to release million-token (1M/2M) context capabilities, the industry has developed a cognitive bias of "long context equals intelligence." Yet in real production testing, unstructured prompts as short as 10,000 characters often cause severe logical collapse. This article breaks down the technical nature of this phenomenon from two dimensions: attention allocation and probabilistic prediction error.

1. Attention Dilution: The Physical Constraint of a 100% Budget

The core of the Transformer architecture is self-attention. Mathematically, the Softmax function normalizes weight allocation over the input sequence so the total sums to 1.

When input length expands from 100 to 10,000 tokens, the attention weight received by each core token is theoretically diluted by two orders of magnitude. This dilution produces the well-known "U-shaped preference" (Lost in the Middle): the model is most sensitive to representations at the beginning and end of the sequence, while critical instructions in the middle positions (e.g., characters 4000–7000) often see their attention scores drop below 0.001%, being effectively ignored in the computation graph. 📉

2. XML Structuring: Building a Firewall for Machine Cognition

Because AI cannot automatically distinguish "background material" from "execution instructions" within verbose natural language, long-form input easily triggers instruction-axis collapse. The industrial-grade solution is an XML tag-based architecture:

Physical isolation: Tags such as <Context>, <Constraints>, and <Workflow> force the model to align features across different semantic blocks in the self-attention layers.
Logical pruning: Remove vague rhetoric like "hopefully" and "if possible," compressing low-entropy natural language into topological nodes of high information density. 🏗️

3. Probabilistic Accumulation and the Collapse of Output Boundaries

A 1M-token context usually refers to input-side KV Cache capacity, not generation capability. AI generation is autoregressive probabilistic prediction: each generated token becomes background for the next step. As output length grows (typically beyond 8,000 tokens), the entropy of the predicted sequence increases exponentially due to accumulated error. This explains why AI falls into loops or produces logical fractures when handling very long outputs. 📈

Conclusion

The essence of million-token context is "storage expansion," not a "comprehension leap." Effective prompt engineering is no longer a rhetorical contest but a systematic compilation process concerning token economics and information density control.

---

References

Research: Stanford University, *"Lost in the Middle: How Language Models Use Long Contexts"*, 2024.
Technical Guide: Deepseek Research, *"Optimization of Agentic Reasoning via XML Boundary Definition"*, 2025.
Case Study: Morii AI, *"First Principles of Prompt Reconstruction for Long-Context Models"*, 2026.