Supermemory's ASMR: How Multi-Agent Memory Retrieval Hit 99% on LongMemEval
> Imagine a friend who reintroduces himself every time you meet. Yesterday's movie chat? Forgotten. Your dietary restrictions? Ignored. That's the predicament of most AI today.
Key points
- ASMR (Agentic Search and Memory Retrieval) is Supermemory's experimental memory system that reported 99% accuracy on LongMemEval, the hardest AI memory benchmark (115k-token histories, multi-session facts, contradictory updates, temporal reasoning questions). Top systems previously struggled to exceed 80%.
- Its core thesis: instead of matching vector similarity, have AI agents actively read and reason over information.
- Extractor: explicit facts ("I like coffee", "I work at Google")
- Profiler: user profile (preferences, habits, traits)
- Chronologist: timeline of events
- Fact Finder: directly relevant facts
- Context Miner: implicit context and associations
- Timeline Reconstructor: event ordering
- Strategy A — 8 parallel experts (counting, temporal, deep-context, etc.): success if any path answers correctly → 98.6%
- Strategy B — 12 agents vote, aggregated by GPT-4o-mini weighing domain trust and answer consistency → 97.2%
- Supermemory made a bold claim: *"Agent memory might be completely solved now."*
- For agents: billions of AI agents currently cannot maintain long-term personalized relationships; memory-capable AI could shift from novelty to utility.
- Methodological shift: sometimes smarter searchers beat smarter indexes—from "retrieval systems" to "understanding systems."
- Open source: full code promised for early April (GitHub).
- Cost: every query involves multiple LLM calls—far pricier than one vector lookup; economics at scale remain unverified.
- Experimental status: ASMR is explicitly a highly experimental pipeline, not yet in Supermemory's production engine.
- Generality: LongMemEval is synthetic; real-world domains (healthcare, legal, finance) may be messier.
- Not the end of vector databases: for massive document retrieval and similarity recommendation, vector search remains the most effective tool. ASMR targets memory scenarios requiring deep understanding and reasoning.
- Supermemory research: https://supermemory.ai/research/
- LongMemEval paper: https://arxiv.org/abs/2410.10813
- Supermemory GitHub: https://github.com/supermemoryai/supermemory
- Dhravya Shah blog: https://supermemory.ai/blog/
The problem: LLMs have "goldfish memory"
LLMs are stateless processors—each session starts from zero. The standard fix, RAG (Retrieval-Augmented Generation), chunks conversation history into vectors stored in a vector database and retrieves the most "semantically similar" chunks at query time. But vector similarity is fuzzy: it handles "looks like" well, yet fails at temporal ordering, information updates, and fact overrides. If a user says "I'm going to Shanghai next Wednesday" then corrects to "Thursday," vector search may return both statements and leave the model to guess.
How ASMR works: a "3+3+multi" agent pipeline
Layer 1: Three Observer Agents (ingestion)
Running on Gemini 2.0 Flash, three parallel agents extract structured knowledge points (not embeddings) along six dimensions—personal info, preferences, events, time data, locations, and implicit context:
Layer 2: Three Search Agents (retrieval)
Instead of querying a vector database, ASMR dispatches:
These agents actively judge which information is newest, resolve contradictions, and infer temporal relationships.
Layer 3: Parallel multi-variant reasoning
Why ASMR wins
1. No vector database: Observer Agents detect information updates at ingestion and maintain version history, so Fact Finder retrieves the latest version rather than contradictory duplicates.
2. Dual timestamps: each memory carries documentDate (when the conversation happened) and eventDate (when the described event happened), enabling temporal reasoning.
3. Relationship version control: memories are linked as updates (corrections), extends (supplements), and derives (inferences), forming a dynamically evolving knowledge graph.
4. In-memory operation: no external vector store; ~50 ms per API call for profile generation; embeddable even in robot hardware.
LongMemEval results
| Category | ASMR result | |-----|-----| | Single-Session User / Assistant | Near-perfect | | Single-Session Preference | Significant improvement | | Knowledge Update | Significant advantage | | Temporal Reasoning | 76.69% (historic weak spot) | | Multi-Session | 71.43% (historic weak spot) |
Temporal reasoning and multi-session integration have long been the weakest categories for vector methods, since similarity cannot express "before/after" and cross-session facts often lack semantic overlap. ASMR's explicit timeline management and active reasoning made breakthroughs here.
Implications and caveats
Caveats to keep in mind: