Agent Memory as a Pyramid, Not a Warehouse: Hierarchical Memory Engineering with TencentDB-Agent-Memory

When you run an Agent on a long task, after about 50 steps it starts to "lose memory" — it forgets what it did earlier, re-invokes the same tools, and the context window fills with verbose logs. You end up re-explaining everything from scratch.

This is not a "cannot remember" problem. With a large enough context window and a comprehensive vector database, everything is technically stored. The real problem is stored but unfindable.

The Dead End of Flat Storage

The traditional approach to Agent memory is intuitive: chop up all historical conversations, tool outputs, and intermediate states, dump them into a vector database, and rely on semantic retrieval at query time.

This is equivalent to piling every document on the floor and squatting down to search when you need something. Every piece of information is an equally important fragment, retrieval becomes a blind exhaustive scan with no macro-level structure to guide it. As the task grows, context balloons, token consumption explodes, and the Agent's attention gets diluted across irrelevant details.

Tencent Cloud's TencentDB-Agent-Memory project (GitHub Trending +1091⭐/day) explicitly rejects this path. Its core thesis is: memory is not about hoarding everything, but about freeing people from repeating themselves.

Two Pillars: Layering + Symbolization

The architecture rests on two pillars: memory layering and symbolic memory. The goal is not for the Agent to "remember more", but for it to "reason better".

Three-Tier Short-Term Memory

Short-term context is not a flat log, but a progressive three-tier structure:

• Bottom tier: raw tool output archives (refs/*.md), preserved in full but excluded from the context window
• Middle tier: step-level summaries in jsonl, recording what each step did and the result
• Top tier: a lightweight Mermaid canvas that compresses the entire task state into a symbolic graph

The Agent only places the top-tier Mermaid structure in its context. When details are needed, it drills down to the middle or bottom tier by node_id. It is like looking at a map: glance at the map to know roughly where you are, then zoom in to the street view when you need detail.

Four-Level Semantic Pyramid for Long-Term Memory

Cross-session long-term memory is also not a flat log, but a four-level semantic pyramid:

• L0 Dialogue: raw conversation
• L1 Atoms: atomic facts extracted from the dialogue
• L2 Scenarios: scenario blocks assembling atoms into situations
• L3 Persona: user profile, abstracting stable preferences from scenarios

Each level is a compression and abstraction of the one below. L3 is not a summary of L0; it is a stable structure refined step by step from L0 → L1 → L2 → L3. This is isomorphic to how human memory is hierarchically processed — short-term memory consolidates into long-term memory, and long-term memory abstracts into "who I am".

Symbolization: From Logs to Symbols

Symbolic memory solves another problem: tool logs are too long. A single tool call may produce thousands of tokens of output, but only a few dozen tokens carry useful information. TencentDB-Agent-Memory compresses verbose tool logs into compact Mermaid symbols, sharply reducing token usage while making the structure clearer.

The Numbers Speak

Measured results after integrating with OpenClaw:

| Benchmark | Horizon | Pass Rate | Relative Gain | Token Usage | Relative Drop | |---|---|---|---|---|---| | WideSearch | short-term | 33% → 50% | +51.52% | 221M → 86M | -61.38% | | SWE-bench | short-term | 58.4% → 64.2% | +9.93% | 3474M → 2375M | -33.09% | | AA-LCR | short-term | 44.0% → 47.5% | +7.95% | 112M → 77M | -30.98% | | PersonaMem | long-term | 48% → 76% | +59% | — | — |

Note the SWE-bench test setup: 50 consecutive tasks simulate the cumulative context pressure of a long-running session, not isolated turns. This better reflects real Agent working conditions.

Engineering Insight: Organization Beats Volume

The core claim — "layering beats flatness" — converges with several recent lines of research:

• MemTools (a USB-C interface for Agent memory): declarative data contracts make different memory systems interchangeable, emphasizing structured organization
• Regression Tax (skill libraries can make Agents worse): 59% of gains are offset by regressions, one root cause being flat retrieval over the skill library
• Zero-Mem (zero-token memory operations): the Agent memory system does subtraction, isomorphic to TencentDB's "do not hoard"

These efforts point to a shared theme: in the Agent era, the key to memory systems is not capacity, but organization.

Analogy: From File Pile to Library

• Traditional vector store = pile every document on the floor, search when needed
• Layered storage = filing cabinet + index + labels, look at the index first, then drill down
• Short-term Mermaid canvas = a map, look at the map first, then walk
• Long-term semantic pyramid = a library classification system (Dewey Decimal), from coarse to fine

This "hierarchical organization" principle is not new — human libraries have practiced it for two millennia. What is new is bringing it into the LLM era, so that Agents retrieve their own memories like librarians rather than digging through nut piles like squirrels.

Conceptual Lineage: Solving the Problem at a Different Layer

TencentDB-Agent-Memory belongs to the conceptual family of "solving the problem at a different layer" — instead of doing memory retrieval harder (bigger vector store, longer context), it switches the layer (hierarchical organization + symbolic compression). It shares the same family of thinking as octopus RNA editing, slime mold externalized memory, and Möbius RoPE topological intervention.

Conclusion

In the arms race of "bigger context, longer window", TencentDB-Agent-Memory suggests another possibility — the competitiveness of a memory system lies not in capacity but in organization. Let the Agent remember what should be remembered, forget what should be forgotten, and free people from repetitive labor so they can focus on judgment, creation, and what truly matters.

---

Project: https://github.com/TencentCloud/TencentDB-Agent-Memory Docs: the README ships with full Chinese and English versions plus a Quick Start License: MIT