Mem++: Don't Compress Memory at Write Time—Leave the Choice to Query Time

When Your Organization's Memory Gets "Compressed" Too Early

Imagine you're an AI assistant responsible for answering questions about a company's decisions. Someone asks: "Which vendor did we pick in March?"

It sounds simple, but inside an organization, "which vendor in March" might mean:

• Feb 15: Engineering sends a memo recommending Vendor A
• Mar 3: Procurement emails that Vendor B was selected
• Mar 20: The CEO signs off, reverting to Vendor A
• Apr 1: A new update arrives selecting Vendor C

If you only store "the latest decision," you can't answer "who had we picked as of March 15?" If you compress each document into a fact like vendor = B at write time, you lose the context of who said what and when.

Mem++'s core insight: most memory systems make a mistake—they compress documents into facts at write time, but no one has asked a question yet, so you have no idea which information will matter.

Write-Time Compression vs. Read-Time Selection

Mainstream LLM memory systems—RAG, knowledge graphs, MemGPT-style hierarchical memory—share a common assumption: the moment a document enters memory, it should be "digested" into structured facts, notes, or graph edges.

This seems reasonable: compressed storage is cheap and retrieval is fast. But the Mem++ authors point out that this amounts to locking in what questions can be answered before the question is asked.

Analogy: you walk into a library looking for books on WWII. One library has shredded every book into index cards—"WWII began in 1939," "D-Day was 1944"—and you can only find these pre-set facts. Another library keeps every book intact, and when you ask a specific question, it retrieves the relevant passages.

The first is write-time compression; the second is read-time selection. Mem++ chooses the latter.

Mem++'s Three-Stage Architecture

Mem++'s design is extremely simple, in three stages:

Write path: Each document is stored verbatim, with date and author attached. No generative model is called at write time—no summaries, no entity extraction. Storage cost is just the raw document itself.

Optional consolidation layer: Near-duplicate documents are grouped, and older versions are marked "superseded"—but not deleted. Consolidation only adds relationship annotations; the original records are fully preserved.

Read path: When a question arrives, the system retrieves only documents from before the question's point in time, then fuses lexical and semantic ranking to select relevant evidence. The choice is handed to the answering model—it can see multiple versions of a document and judge for itself which one was valid at the time the question refers to.

The key distinction: Mem++ isn't simply "keeping everything" (that's just a database)—it keeps every document intact, attaches dates, and defers version selection to read time.

What "Non-Destructive" Actually Means

"Non-destructive" sounds like jargon, but it has a concrete meaning:

1. No overwriting: New documents don't overwrite old ones. If a March 20 memo changes the vendor choice, the March 3 email is still fully preserved. 2. No distillation: A 10-page memo is never compressed into a triple like vendor = B. A document is a document. 3. No generation at write time: No LLM is invoked for summarization or entity extraction. The write path is a pure storage operation—zero token consumption.

Together, these three mean Mem++'s write path is extremely cheap—it's just a document store with a date index.

Results: 8–13 Point Gains

Mem++'s performance on OrgMemBench (an organizational memory benchmark):

• Outperforms the strongest baseline by 8.0–13.1 points (across two answering models)
• With gpt-4.1-mini, scores 2.6 points above RAG overall
• Best LLM-judge average score on LoCoMo (a conversational benchmark)
• Second place on LongMemEvalS, behind only its own entity-graph variant

These gains don't come from a smarter LLM or a more complex architecture, but from a simple design choice: don't compress at write time.

Why This Is Counterintuitive

Mem++ seems to "waste" storage. Keeping every document in full—no compression, no distillation—isn't that more expensive?

But the authors argue the real cost isn't storage, it's information loss. When you compress a memo into vendor = B, you lose:

• Who wrote the memo
• When it was written
• What the context was
• Whether there were conditions attached
• Whether other options were mentioned

This information looks unimportant at write time, but at read time it may be exactly what's needed to answer the question. The price of compression isn't storage space—it's the freedom to answer future questions.

A Deeper Insight: Choice Itself Has Value

Mem++ evokes a concept from finance: option value.

A financial option has value because it gives you the right—not the obligation—to make a choice at a future point. You don't need to know today whether a stock will rise or fall; you just need to keep the option alive.

Memory systems are the same. Write-time compression is like "exercising early"—you make a choice about what matters and what can be discarded without ever seeing the question. That forfeits the option to answer future queries differently.

Mem++'s philosophy: keep all documents, defer selection to read time. In finance this is "delayed exercise"; in memory systems, "non-destructive writes."

Implications for Agent Systems

Agent memory research has a visible trend: increasingly sophisticated write-time processing. MemGPT lets the LLM decide when to compress; Hindsight learns mental models from interactions; AutoCompact jointly trains compression decisions with task rewards.

All of these try to make "compression" smarter. Mem++ takes the other road: don't compress—just store.

This doesn't mean compression is always wrong—for pure conversation scenarios (LoCoMo, LongMemEval), compression remains competitive. But for organizational memory—multiple people, multiple documents, multiple versions—the advantage of non-destructive writing is decisive.

When information comes from a multi-author document stream rather than a single conversation history, the information loss from write-time compression is too large.

A More Fundamental Question

Mem++ exposes a deeper question: why do we default to compression at all?

Perhaps because the RAG paradigm is so deeply ingrained—"retrieval-augmented generation" implies retrieving compressed knowledge chunks. Perhaps because storage was historically the bottleneck, so "compression" became a default good habit.

But in 2026, storage is no longer the main constraint; LLM context windows have grown from 4K to 128K and beyond. What's truly scarce isn't storage space—it's information completeness.

Mem++'s contribution isn't a new algorithm; it's questioning a widely accepted default assumption. Sometimes the best innovation isn't "doing things more cleverly"—it's *not doing them*.

---

Paper: Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents

Code: github.com/AIDAChip-Inc/mem-plus-plus

Benchmark: github.com/JackCGardner/OrgMemBench