> First, answer me one thing—if an AI writes code for you but greets you like a stranger every single time, is that what you want?
I spent several days digging into agentmemory, a project by founder Rohit that claims to give AI coding assistants long-term memory. 3,400 GitHub stars, built in two months. The website's numbers are pretty: 95.2% retrieval accuracy, at a cost of about $10 per year.
But after seeing enough so-called "breakthroughs," I've learned one thing: pretty numbers don't prove anything. What matters is whether you can explain what this thing actually does in words a sixth-grader would understand.
So let's start from scratch.
---
1. The Problem: Why Do AI Agents Need Long-Term Memory?
Imagine a coding partner who is highly capable, but every time you approach him, he's completely forgotten what you discussed. The bug you spent three hours debugging yesterday? "What bug?" The architecture decisions you explained for half an hour? "Why didn't you tell me earlier?"
That's the real state of today's AI coding assistants—Cursor, Copilot, Claude Code. They have no memory. Or rather, their memory is only the context window of the current conversation.
Karpathy (former Tesla AI director, OpenAI founding member) captured the problem in an April note:
> Stop re-deriving, start compiling.
His point: today's AI has to "understand" you from zero every time—your code style, project structure, preferences. It's like re-deriving the fundamental theorem of calculus before every exam. Why not keep a notebook—a "Wikipedia" the AI can just open and use?
That's what agentmemory tries to solve. Not making AI smarter—making AI remember.
---
2. How Does agentmemory Solve It?
1. Automatic capture: 12 "hooks"
Past memory systems required you to manually say "remember this"—tagging frame by frame: "remember I hate TypeScript." But if something requires constant human effort, it will eventually be forgotten. Humans get lazy.
So agentmemory has 12 automatic capture hooks—essentially instrumentation. Every terminal command, tool call, file edit gets recorded without you saying "remember." No habit change required; memory accumulates quietly in the background.
2. Retrieval: three methods at once
Recording is only step one. The key is finding things back.
agentmemory mimics how the brain recalls—sometimes by keyword, sometimes by fuzzy concept, sometimes by association. It retrieves three ways simultaneously:
- BM25 — keyword matching, like a search engine
- Vector search — semantic similarity, finding "same meaning, different words"
- Knowledge graph — entity relationships, e.g., "this file relates to that bug"
- Working memory — the current moment, e.g., the last command
- Episodic memory — past events, e.g., "last Wednesday we fixed that bug"
- Semantic memory — abstract knowledge, e.g., "the project uses React, not Vue"
- Procedural memory — automated habits, e.g., "the user always writes tests first"
- LoCoMo: 91.6%
- LongMemEval: 93.4%
- BEAM(1M): 64.1%
- Paper A: 66.9%
- Paper B: 68.4%
- Mem0's own old algorithm: 71.4%
- Mem0's own new algorithm: 91.6%
- 12 automatic capture hooks solve the real problem that humans won't remember to tag manually
- Triple retrieval (BM25 + vectors + knowledge graph) is a sound engineering decision
- Four-tier memory consolidation has cognitive-science grounding, not armchair thinking
- ~$10/year makes it affordable for individual developers
- The mem0 81.4% in the comparison table is likely outdated
- The "92% token reduction" is measured against a strawman
- The benchmarks' inherent instability undermines any precise claim like "95.2%"
- Mem0's new algorithm has shrunk the gap to near-irrelevance
Then it fuses and re-ranks all three results (RRF) to pick the best.
3. Layered memory: organizing like a human brain
The most interesting design. agentmemory divides memory into four tiers:
Inspired by how the brain consolidates memory during sleep: messy daytime information gets organized and archived. agentmemory similarly distills scattered fragments into more durable knowledge.
4. The foundation: the iii Engine
agentmemory isn't built on the usual Express.js + SQLite. It runs on the iii engine, a brand-new runtime Rohit wrote himself—51 MCP tools, 873 tests, plus a live viewer on port 3113 that can replay your sessions.
This is a bet. Either the iii engine becomes next-generation infrastructure, or agentmemory gets locked onto a runtime nobody maintains.
---
3. The Numbers: Is 95.2% Real?
The comparison table on the site:
| System | Accuracy | |--------|----------| | agentmemory | 95.2% | | mem0 | 81.4% | | Letta | 73.8% |
Pretty, right? But: under what conditions were these measured? Same questions, same judging criteria?
Problem 1: What's the benchmark?
agentmemory claims 95.2% R@5 and 98.6% R@10 on LongMemEval-S (ICLR 2025, 500 questions). But LongMemEval-S itself has about 5% label noise—some "correct answers" may simply be wrong. 95% is already near the theoretical ceiling.
Problem 2: Apples-to-apples?
The table puts agentmemory's 95.2% next to mem0's 81.4%. But which version, which configuration of mem0 is that 81.4% from? Mem0's new algorithm (April 2026) scores:
Mem0's new algorithm scores 93.4% on LongMemEval, not 81.4%. That's like saying "my car is faster than yours" without specifying highway or racetrack.
Problem 3: The "92% token reduction"
agentmemory claims ~170K input tokens per year versus 19.5M+ for "pasting full context"—a 92% reduction. But who actually pastes their full context every time? That's a strawman argument. The real comparison should be against other memory systems' token consumption, not an extreme approach nobody uses.
Problem 4: Wildly different numbers for the same system
Mem0's reported LoCoMo scores:
Same system, same benchmark, numbers jumping from 66% to 91%. Mem0's own blog admits scores carry a ±1 confidence interval because judging criteria are inconsistent. If the ruler itself is unstable, any precise claim like "95.2%" deserves a question mark.
Problem 5: The embedding model
agentmemory uses all-MiniLM-L6-v2—a lightweight, free, locally-run model. Good for cost. But its embedding quality falls short of systems using GPT-4 or Claude. Part of the number advantage comes from "I achieved similar results with cheaper tools," not "my architecture is inherently superior."
---
4. How Does It Compare to Alternatives?
mem0: 51.8K stars, the veteran
The most mature memory system, but with one essential difference: it requires explicit add() calls. You have to write mem0.add("user says he doesn't want popup notifications"). If something needs an extra manual step, people forget. agentmemory's automatic capture wins here. But mem0's new algorithm has caught up—93.4% vs 95.2%—and it has a huge community and mature ecosystem.
Letta: full agent runtime
Letta isn't just memory—it's a complete agent framework. The cost is lock-in: your entire application architecture must revolve around it.
Built-in solutions (CLAUDE.md)
Claude's CLAUDE.md holds only ~200 lines and everything gets stuffed into context. Blunt, but limited.
agentmemory's real advantages
Not the 14-point gap. Instead:
1. Zero-config automatic capture — no code changes, no API calls, just work normally 2. Karpathy's philosophy — from "re-deriving every time" to "compile once, reuse forever" 3. Brain-inspired architecture — triple retrieval + four memory tiers, with real design thinking
---
5. iii Engine: Bet or Shackles?
agentmemory runs on a brand-new runtime—not Node.js, not Django—something Rohit built himself. This reminds me of what Feynman saw in the Challenger investigation: management estimated a 1/100,000 failure probability; engineers privately said 1/100. Management believed the numbers they wanted to believe.
The iii engine's problem: if Rohit stops maintaining it, or the community doesn't follow, what happens to agentmemory?
That's a real lock-in risk. On the other hand, the Express.js + SQLite + pm2 + Prometheus stack is itself aging. The iii engine might be a future-proof bet or premature optimization. I'm not sure. The only certainty: a two-month-old project on a brand-new runtime needs time to prove its reliability.
---
6. Growth Rate
Created in late February, now 3,400 stars. For a two-month-old project, that's genuine community interest, not inflation. But stars don't equal quality. Stars mean "someone finds it interesting," not "this actually works well."
---
7. Final Verdict: Hype or Substance?
Feynman would say: I only believe what I can directly verify.
agentmemory's architecture has substance:
But the benchmark numbers have水分—er, padding:
agentmemory's value is turning Karpathy's philosophy into runnable code:
> Stop re-introducing yourself to me every time. Compile what you know about me, and load it next time.
That's directionally correct. Whether agentmemory ultimately wins, the direction is right.
---
8. A Question Feynman Would Ask
If you deleted every benchmark number, every comparison table, every "95.2%"—leaving only code and architecture docs—would it still have value?
My answer: Yes. Automatic capture + triple retrieval + layered memory have real design depth on their own.
But if only the numbers and marketing copy remained, without the architecture?
That would be cargo cult—bamboo control towers, coconut-shell headphones. All the form, but the planes never come.
agentmemory is not cargo cult. It has real design. But those pretty numbers—95.2%, 98.6%, 92% reduction—apply a discount before you read them.
What matters isn't how many points it beats competitors by. What matters is that it gives AI assistants a "memory" for the first time, without requiring you to do anything extra.
That's the way it is.
---
*Sources: agentmemory.dev, Karpathy gist (Apr 2026), Mem0 blog (Apr 2026), LongMemEval (ICLR 2025), LoCoMo benchmark*