Context Engineering: An Architectural Blueprint for Building Cognitive AI Systems

Key points

1. Paradigm shift: from prompt engineering to context engineering

• LLMs are stateless: every API call starts fresh. Early workarounds injected everything into system prompts, but this hits an "attention budget" wall.
• Anthropic defines context engineering as the set of strategies for curating, maintaining, and optimizing all tokens entering the context window during inference; Google DeepMind frames it as dynamically assembling context to build stateful, intelligent agent systems.
• The core philosophy is actuarial management of "cognitive assets" — like managing a kitchen supply chain rather than writing a single recipe.

| Dimension | Prompt Engineering | Context Engineering | |---|---|---| | Core goal | Optimize single-turn output | Optimize long-horizon system coherence | | Object | Text instructions | Full information flow and system state | | Mindset | Creative writing | Systems architecture | | Scope | Single input-output pair | Memory, tools, RAG, user profiles, environment | | Scalability | Hard to scale, manual tuning | Designed for scale, automated ETL pipelines | | Lifecycle | One-off | Continuous, iterative |

2. Context physics: window mechanics and performance decay

• Context rot: retrieval accuracy drops as distractors accumulate; semantically similar distractors create attention competition (needle-in-a-haystack tests). Larger windows (up to 2M tokens in Gemini 1.5 Pro) do not translate proportionally into processing capability.
• Lost-in-the-middle: models attend most to the beginning and end of context (primacy/recency effects), forming a U-shaped performance curve.
• Practical implications: system instructions go first, user queries last; use re-ranking (e.g., LangChain's LongContextReorder) to place high-relevance documents at both ends. Compression (summarization, key extraction) maximizes signal-to-noise within the attention budget.

3. Session layer: the dynamic workbench

• Google's metaphor: a session is a "workbench" — tools and materials at hand for the current task, cleaned up afterward.
• Sessions are containers of time-ordered events (user inputs, replies, tool calls/outputs) plus structured state (task variables, akin to LangGraph's StateGraph).
• Management strategies:
• Sliding window / intelligent truncation: keep recent N turns while preserving system instructions (e.g., Google ADK's ContextFilterPlugin).
• Recursive summarization: compress older history into a summary prepended to recent turns; preserves semantic threads for long tasks.
• Structured note-taking: the agent actively maintains an external XML/JSON file of conclusions and todos, re-injected each turn (Anthropic's recommendation).
• Production concerns: low-latency stores (Redis, Memcached, Firestore) on the hot path; strict per-user/tenant isolation via ACLs; PII scanning before persistence (e.g., Google Cloud Model Armor) for GDPR/CCPA compliance.

| Strategy | Pros | Cons | Best for | |---|---|---|---| | Sliding window | Low latency, simple | Loses early context | Simple chats | | Recursive summarization | Preserves long-term semantics, saves tokens | Lossy details | Long consultations, role-play | | Structured state | High precision, logic-friendly | Needs predefined schema | Task-oriented dialogs | | Note systems | Flexible, accumulative | Adds inference load | Research, multi-step planning |

4. Memory architecture: the foundation of long-term cognition

• Memory vs. RAG: RAG is a "research librarian" for shared, relatively static world knowledge; memory is a "private assistant" for user-specific, frequently updated knowledge. Good agent architectures include both.
• Three-tier cognitive taxonomy:
• Semantic memory ("knowing what"): facts and preferences, built via entity/relation extraction.
• Episodic memory ("remembering when"): timestamped records of specific past events, enabling continuity and causal reasoning.
• Procedural memory ("knowing how"): learned skills and workflows — e.g., Voyager stores successful Minecraft behaviors as reusable code skills; Reflexion stores self-reflections as negative constraints. Research shows procedural memory distilled from strong models (GPT-4) can boost weaker models.
• Memory generation is an ETL pipeline: ingestion of raw logs → LLM-driven extraction (few-shot prompts define "value") → consolidation (deduplication, conflict resolution with timestamp-based updates, synthesis of low-level observations into higher-level reflections, per Stanford's Generative Agents) → storage as embeddings or graph nodes.
• Retrieval combines relevance (vector similarity), recency (exponential decay), and importance (LLM-scored at write time). "Memory-as-a-Tool" (Google ADK) lets agents actively decide when to record or search memory instead of passive retrieval every turn.

5. Frameworks: Google ADK vs. LangGraph

• Google ADK: code-first, event-driven, cloud-native. Treats context as a "compiled view" assembled by processors from persistent sessions. Offers the A2A protocol for multi-agent interoperability (.well-known/agent.json discovery, RESTful task delegation) and deterministic Sequential/Parallel orchestration patterns.
• LangGraph: state-machine/graph paradigm with a shared state object flowing between nodes; natively supports cycles (e.g., Reflexion loops), checkpointing for human-in-the-loop approvals, and time-travel debugging.

| Feature | Google ADK | LangGraph | |---|---|---| | Metaphor | Event pipeline | State machine / graph | | State | Event logs + memory objects | Shared state schema | | Control flow | Modular code (Sequential/Parallel) | Explicit/conditional edges | | Cycles | Implicit (LoopAgent) | Native, explicit | | Multi-agent | A2A protocol (HTTP/REST) | Subgraphs, in-process | | Ecosystem | Google Cloud (Vertex AI, Gemini) | LangChain, model-agnostic | | Best for | Enterprise services, cross-team workflows | Complex reasoning, adaptive loops |

6. Protocols and interoperability

• Model Context Protocol (MCP) — Anthropic's "USB-C for AI applications" — solves the N×M integration problem via a client-host-server architecture (hosts like Claude Desktop/Cursor; servers exposing capabilities over stdio or HTTP-SSE). Three primitives: Resources (readable data), Tools (executable functions with side effects), Prompts (interaction templates). Write once, plug into any MCP-capable AI app.
• Agent-to-Agent (A2A) protocol standardizes agent-to-agent social interaction: Agent Card metadata discovery and task delegation via standardized interfaces, enabling loosely coupled, distributed agent networks.

7. Security and governance

• New threat vectors: (indirect) prompt injection hidden in retrieved documents or memories, context poisoning that pollutes memory bases with false information, and sensitive-data leakage through long-lived memory.
• Google Cloud Model Armor implements defense-in-depth: input sanitization (detecting injection, malicious URLs, PII templates) and output auditing (blocking or masking unsafe responses), plus configurable confidence thresholds, enterprise-specific PII templates, and full audit logging.

8. Conclusion and outlook

Context engineering marks the move from "alchemy" (prompt crafting) to industrial engineering: sessions as workbenches, memory as archives, MCP as connectors, and security layers as shields. The future points toward metacognition — agents that evaluate and manage their own memories and autonomously refine their procedural scripts — plus multimodal contexts enabling long-lived, coherent cognitive presence.

*Key references: Anthropic's "Effective context engineering for AI agents", Google DeepMind's context engineering whitepaper, Chroma's Context Rot research, the Lost-in-the-Middle paper (TACL), Voyager and Reflexion papers, Google ADK and Model Armor documentation, LangChain memory blog.*