Architecting the Agent Mind: Context Engineering with Sessions and Memory

This post is an interactive web application synthesizing the paper "Context Engineering: Sessions, Memory" by Kimberly Milam and Antonio Gulli (Google, Nov 2025). It explores the engineering trade-offs between ephemeral Sessions and persistent Memory for building robust, hallucination-resistant GenAI agents.

Key points

• Sessions (short-term memory): Capture the immediate interaction history. The core challenge is managing the context window — as conversations grow, full history becomes expensive and prone to the "Lost in the Middle" phenomenon.
• Memory (long-term storage): Structured, scalable, and essential for personalization beyond transient sessions.

Context window strategies (Token usage vs. conversation turns)

| Strategy | Token Cost | Context Quality | Notes | |---|---|---|---| | Full History | Grows linearly/exponentially, hits cost & latency limits | Perfect initially, degrades on massive prompts | Optimization is mandatory for production | | Sliding Window | Caps at last N turns (flat) | Drops rapidly as early details are lost | Cheapest, but forgets conversation start | | Summarization | Grows slowly, controlled | Stays high; key insights preserved in summaries | Best cost/recall balance |

Memory taxonomy (modeled after human cognition)

• Episodic Memory — past experiences and event sequences; provides continuity over time.
• Storage: Vector Database + Timestamp / time-decay weighting; retrieval via similarity search biased by recency.
• Example: recalling "we discussed Python decorators last Tuesday."
• Semantic Memory — facts, concepts, and world knowledge.
• Procedural Memory — implicit "how-to" knowledge (e.g., skills and workflows).

Each memory type requires a specific storage architecture (vector stores, knowledge graphs).

Memory generation pipeline

1. Raw Input — user prompts and conversation history. 2. Extraction — identify entities, intent, and facts. 3. Consolidation — de-duplicate and merge with existing knowledge graphs. 4. Storage — persist as vector embeddings and knowledge graphs.

References cited in the post

• Google Vertex AI Documentation
• LangChain / LangGraph Memory Concepts
• Generative AI Agent Architecture

*Note: the original post is an interactive HTML app (Tailwind CSS + Chart.js); the interactive charts and click-through panels are summarized above in table and list form.*