Why Agent Memory Is a Pyramid, Not a Warehouse
A long-running agent often appears to 'lose its memory' after 50 steps: it forgets prior actions, repeats tool calls, and stuffs its context window with verbose logs. This is not a capacity problem. The context window and vector store are large enough; the issue is that the agent remembers everything but cannot find what matters.
The Dead End of Flat Storage
Traditional agent memory systems intuitively dump all historical dialog, tool outputs, and intermediate states into a vector database and retrieve by semantic similarity. This is equivalent to piling every document on the floor and rummaging through it on demand. Every entry is treated as equally important, retrieval becomes indiscriminate global search, there is no macroscopic structure to guide it, and long tasks explode in tokens while the agent's attention dilutes across irrelevant details.
Tencent's TencentDB-Agent-Memory project (trending on GitHub at +1091 stars/day) explicitly rejects this approach. Its core thesis: memory is not about hoarding everything, but about freeing people from repeating themselves.
Two Pillars: Layering and Symbolic Compression
The architecture rests on memory layering and symbolic memory. The goal is not 'remembering more' but 'reasoning better'.
A Three-Layer Short-Term Architecture
Short-term context is not a flat log but a progressive three-layer structure:
- Bottom layer: raw tool output archives (
refs/*.md), fully preserved but excluded from the live context window. - Middle layer: step-level summaries (
jsonl) recording what each step did and what it produced. - Top layer: a lightweight Mermaid canvas compressing the entire task state into a symbolic graph.
- L0 — Dialog: raw conversation.
- L1 — Atoms: atomic facts extracted from the dialog.
- L2 — Scenes: scenario blocks assembling atomic facts into situations.
- L3 — Persona: a user profile abstracting stable preferences from scenes.
- MemTools (a USB-C interface for agent memory): declarative data contracts make memory systems interchangeable, emphasizing structured organization.
- Regression Tax (skill libraries can make agents worse): 59% of gains are canceled by regressions, and flat skill retrieval is a root cause.
- Zero-Mem (zero-token memory operations): agent memory systems that subtract rather than add, isomorphic to TencentDB's 'do not hoard' stance.
- Flat vector store = files stacked on the floor, rummaged on demand.
- Layered store = a cabinet with directories and labels, look up the index first, then drill down.
- Short-term Mermaid canvas = a map: orient first, then navigate.
- Long-term semantic pyramid = a library classification system (Dewey decimal), locating items from coarse to fine.
- The dead end of flat storage
- Two pillars: layering and symbolic compression
- A three-layer short-term architecture
The agent only keeps the top-layer Mermaid structure in its context. When detail is needed, it drills down through node_id to the middle or bottom layer. The analogy is a map: consult the map for orientation, then zoom into street-level detail.
A Semantic Pyramid for Long-Term Memory
Cross-session long-term memory is also not a flat log but a four-tier semantic pyramid:
Each layer is a compression and abstraction of the layer below. L3 is not a summary of L0 but a stable structure distilled through L0 → L1 → L2 → L3. This mirrors the hierarchical consolidation seen in human memory: short-term memory stabilizes into long-term memory, which abstracts into a sense of identity.
Symbolization: Turning Logs into Symbols
Symbolic memory tackles a different problem: tool logs are too long. A single tool call can emit thousands of tokens of output while only a few dozen carry real signal. TencentDB-Agent-Memory collapses verbose tool logs into compact Mermaid symbols, dramatically reducing token usage while keeping the structure clearer.
What the Numbers Say
Integration with OpenClaw produced these measured results:
| Benchmark | Horizon | Pass Rate | Relative Gain | Token Usage | Relative Drop | |---|---|---|---|---|---| | WideSearch | short | 33% → 50% | +51.52% | 221M → 86M | -61.38% | | SWE-bench | short | 58.4% → 64.2% | +9.93% | 3474M → 2375M | -33.09% | | AA-LCR | short | 44.0% → 47.5% | +7.95% | 112M → 77M | -30.98% | | PersonaMem | long | 48% → 76% | +59% | — | — |
Note the SWE-bench setup: 50 sequential tasks simulate context-accumulation pressure across a long session rather than isolated turns, matching real agent workloads.
Engineering Insight: Organization Beats Volume
The thesis that layering beats flatness converges with several recent works:
Together these studies point to the same theme: in the agent era, the competitive axis for memory systems is organization, not capacity.
Analogy: From Pile to Library
The principle of layered organization is not new — human libraries have practiced it for two millennia. What is new is bringing it into the LLM era so that agents retrieve their own memories like librarians rather than foraging like squirrels.
Conceptual Lineage: Solving by Changing the Layer
TencentDB-Agent-Memory belongs to a 'solve by changing the layer' family — instead of doing memory retrieval harder (bigger vectors, longer context), it shifts to a different level (hierarchical organization plus symbolic compression). It is part of the same conceptual family as octopus RNA editing, slime-mold externalized memory, and Möbius RoPE topological intervention.
Closing
In the arms race toward ever-larger context windows, TencentDB-Agent-Memory suggests an alternative: the competitiveness of a memory system lies in organization, not capacity. Let agents remember what should be remembered, forget what should be forgotten, and free people to focus on judgment, creation, and the work that truly matters.
Project: https://github.com/TencentCloud/TencentDB-Agent-Memory Docs: bilingual README with Quick Start License: MIT
FAQ
Q1. Who is this for? Practitioners, researchers, and students interested in AI, machine learning, and deep learning.
Q2. What are the core takeaways?