Context of Context: A Cognitive Revolution in AI from Stone Tools to Starships — Review of 'Context Engineering 2.0'

Forum topic · ✨步子哥 · 2025-11-18

Summary

A Chinese tech forum review of the 2025 arXiv paper 'Context Engineering 2.0: The Context of Context Engineering' (arXiv:2510.26493). The paper formalizes context engineering as an entropy-reduction process—converting high-entropy, noisy user signals into structured, machine-usable representations—and maps four eras of context engineering: Era 1.0 (sensor-driven primitive computing, 1990s–2020), Era 2.0 (LLM-era agentic intelligence, present), Era 3.0 (human-level mutual understanding, projected 5–15 years), and Era 4.0 (superhuman intelligence where machines proactively build context about users). The post summarizes the paper's three operational dimensions—collection (multimodal sensing up to brain-computer interfaces), management (self-baking memory, short-to-long-term transfer, compression), and usage (internal/inter-system sharing, context selection, proactive user-need inference). It highlights formal definitions, memory equations, challenges like Transformer O(n²) scaling and attention dilution, and future directions, arguing prompt engineering is giving way to context architecture.

Context of Context: A Cognitive Revolution in AI from Stone Tools to Starships

Imagine standing before ancient cave paintings, where early humans used crude lines to depict prey and stars—the earliest "context": capturing the essence of an infinite world with limited symbols. Fast forward to 2025: AI no longer passively waits for instructions but works like a tireless detective, piecing together your past, present, and potential future. The source of all this is the paper that shook the AI world—Context Engineering 2.0: The Context of Context Engineering (arXiv:2510.26493). After reading it, you realize that all our previous "AI tuning"—carefully polished prompts, mountains of RAG retrieval, fragile tool calls—were stone-age axes, and the true interstellar age has quietly begun.

From Marx to Machines: A Striking Parallel

The paper opens with a bombshell: citing Marx's famous line that "the essence of man is the ensemble of social relations," then coldly adding—might the essence of a machine be the totality of its contexts?

> Context is defined as: any information that can be used to characterize the situation of entities relevant to the interaction between a user and an application. It includes not just conversation history but environmental noise, emotional fluctuations, historical memory, even unspoken intent. High-entropy signals from messy daily life must be converted through engineering into low-entropy structured representations.

Humans are natural low-entropy communicators—when chatting with a friend, you never re-explain why you're feeling down; they've auto-filled the gaps from your tone and last week's complaints. Machines must be engineered to do the same. The paper formally defines context engineering as an entropy-reduction process:

$ \text{Char} : E \to \mathcal{P}(F) $ (entity-to-feature mapping)

$ C = \bigcup_{e \in E_{\text{rel}}} \text{Char}(e) $ (context is the union of features of relevant entities)

$ \text{CE} : (C, T) \to f_{\text{context}} $ (context engineering maps raw context C and task T to an optimized function)

These formulas reveal the core driving force of AI evolution: the more intelligent the machine, the larger the context it can process, driving human-interaction cost toward zero.

The Four-Act Drama of AI Civilization

The paper's most exciting part is an evolutionary model spanning 30 years (Figure 1, currently transitioning from 2.0 to 3.0):

Era 1.0: Primitive Computing (1990s–2020) — Clunky GUI menus and Context Toolkit systems. Humans acted as "intent translators," decomposing thoughts into clicks, dropdowns, forms. Context collection relied on single-modality sensors (location, temperature); interaction cost was extremely high.
Era 2.0: Agent-Centric Intelligence (2020–present) — The LLM explosion (ChatGPT, LangChain, Claude Projects) lets systems understand natural language. We upgraded from translator to "context architect," but we're still propping things up with conversation-history stacking: compress, RAG, tool-call whenever the context window bursts—like building a 3.0 skyscraper with 2.0 bricks.
Era 3.0: Human-Level Intelligence (next 5–15 years?) — Machines become "equal partners" that understand you without explanation, collecting emotional and social cues via full multimodal perception. Interaction cost approaches natural conversation.
Era 4.0: Superhuman Intelligence (further future) — The scariest prediction: machines build context about *us*. Detecting elevated heart rate, salary-cut news browsing, low mood, it proactively pushes career-transition plans, psychological intervention, side-hustle paths. Interaction cost turns negative—AI satisfies unexpressed needs in advance.

Figure 2 shows carbon-based and silicon-based cognitive trajectories crossing: human intelligence climbs slowly while machines explode exponentially.

Collection: From Passive Ingestion to Multimodal Sensing

In Era 1.0, collection was sensor-dominated and mechanical. Era 2.0 added multimodality—text, images, audio, wearable heart-rate data. The paper criticizes the dumbest practice: remembering everything the user says until the context window balloons.

> The correct approach follows Minimal Sufficiency and Semantic Continuity—your AI's memory should be a smart butler, not a garbage bin.

Era 3.0 collection? EEG, eye tracking, skin conductance, real-time hormone streams—Neuralink is just the appetizer. Table 2 lists representative collectors, from smartphones to brain-computer interfaces.

Management: Self-Baking—AI Learns Human-like Forgetting and Memory

Management is the paper's densest battlefield, covering text processing, multimodal fusion, organization, and abstraction. It criticizes naive truncation, recommending a "sandwich structure" of QA pairs, hierarchical notes, and vector summaries. The most brilliant concept is Self-Baking—letting AI decide when to summarize and when to discard, like humans converting episodic to semantic memory:

$ M_s = f_{\text{short}}(c \in C : w_{\text{temporal}}(c) > \theta_s) $ (short-term memory: high temporal relevance)

$ M_l = f_{\text{long}}(c \in C : w_{\text{importance}}(c) > \theta_l \wedge w_{\text{temporal}}(c) \leq \theta_s) $ (long-term memory: high importance)

$ f_{\text{transfer}} : M_s \to M_l $ (memory transfer)

This layered architecture (Figure 6)—natural-language summarization, fixed-pattern extraction, vector compression—is a silicon-based replica of human forgetting. Like a hippocampus, AI quietly "bakes" the day's experiences overnight. Multimodal processing uses hybrid strategies (Figure 5): vector mapping, self-attention, cross-attention. Organization includes context isolation—sub-agents with separate memory, like corporate department walls.

Usage: From Passive Responder to Active Symbiont

Six sub-practices:

Internal sharing: prompt embedding, structured messages, shared memory (Figure 7).
Cross-system sharing: adapters, unified representations.
Context selection: semantic relevance + logical dependency + recency/frequency priority.
Proactive user-need inference: the nuclear option! AI notices you rewriting the same code three times and pops up: "Shall I refactor the architecture directly?" It's no longer an assistant but a digital symbiont.
Lifetime preservation: dynamic updates, like a semantic operating system.
Emerging practices: KV-cache optimization, tool design (fewer than 30 tools to avoid choice paralysis).

Examples abound: Gemini CLI auto-searching docs, Claude Projects' sub-agent isolation, Tongyi DeepResearch's periodic summarization. These products were unconsciously practicing subsets of Context Engineering 2.0—now they have a "theoretical mother"!

Application Scenarios

CLI tools: context includes user, environment, tools, memory.
Deep Research: periodic compression of history into reasoning states.
Brain-computer interfaces: capturing neural signals directly, skipping language as an inefficient intermediary.

Challenges and Future

The paper is honest about pain points: storage bottlenecks, processing degradation (Transformer's O(n²) curse), system instability, evaluation difficulty... Under long contexts, models even suffer attention dilution on middle tokens. Future directions: natural multimodal collection, scalable storage, enhanced understanding, new architectures (replacing Transformer), adaptive selection, digital presence. The paper predicts context will become our persistent identity, and AI will surpass humans at understanding intent.

Conclusion: Welcome to the Interstellar Age of Context Engineering

Reading this paper felt like being struck by lightning—all our agent projects so far have been smashing nuclear reactors with stone tools. From the first principle of entropy reduction, to the four-era transition curve, to the eternal trinity of collection-management-usage, this work isn't a survey—it's a bible. It tells us: prompt engineering is dead; the era of the Context Architect has arrived.

Imagine a few years from now, when your AI companion knows you better than your mother—not just remembering your birthday but preparing solutions to your potential crises in advance. Then we will truly evolve from "using AI" to "co-existing with AI."

> The paper's shock lies not in what it says, but in revealing that after all our busy work, we've only just walked out the door.

--- References

1. Hua, Q. et al. (2025). Context Engineering 2.0: The Context of Context Engineering. arXiv:2510.26493. https://arxiv.org/abs/2510.26493 2. Dey, A. K. (2001). Understanding and Using Context. Personal and Ubiquitous Computing. 3. Liu, P. et al. (2021). Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. 4. Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. 5. Weiser, M. (1991). The Computer for the 21st Century. Scientific American.