Introduction
Have you ever been stuck on a math problem, closed your eyes, and let your brain "find it on its own" — then suddenly a similar example or a technique your teacher once mentioned surfaced like a bubble from underwater? You didn't "read" that memory; you *felt* it. That's implicit memory: it doesn't depend on active recall, it emerges naturally in the gaps of thinking.
AI memory systems have long had only two options:
- Rewrite model weights (fine-tuning) — risking catastrophic forgetting
- Bolt on RAG (retrieval-augmented generation) — stuffing raw text into context, as awkward as inserting an encyclopedia into a conversation
- The main reasoning model is fully frozen — no catastrophic forgetting
- Only the Memory Weaver's LoRA parameters are optimized — lightweight and efficient
- Supports both SFT and GRPO training — supervised learning and reinforcement learning both work
- Title: MemGen: Weaving Generative Latent Memory for Self-Evolving Agents
- Authors: Guibin Zhang, Muxin Fu, Shuicheng Yan (National University of Singapore)
- arXiv: https://arxiv.org/abs/2509.24704
MemGen, proposed by a National University of Singapore team, offers a third path: letting AI generate its own latent memory mid-reasoning. Instead of retrieving external text, it weaves machine-native memory tokens at the moment of thought — natural, immediate, and non-disruptive, like the human subconscious.
The Dilemma of Traditional Memory Approaches
Option 1: Parameter fine-tuning. New knowledge is written into weights, but catastrophic forgetting follows: teach it math and it may forget how to write poetry. The more specialized, the narrower.
Option 2: RAG retrieval. The model stays untouched, but relevant text is retrieved and shoved into context. The result is rigid stitching — retrieved text and current reasoning never organically merge, causing fragmentation and low reasoning efficiency.
MemGen's answer: freeze the main model, train only the memory module.
MemGen's "Subconscious" Architecture
The core architecture has just two modules:
Memory Trigger — when to recall what
A vigilant system that continuously monitors the main model's hidden states. Key design: it triggers only at semantic nodes — periods, semicolons, logical turning points. Rather than asking "should I recall something?" at every token, it judges in the gaps of thought whether an experience is needed. This avoids both extremes: never triggering (no memory) and over-triggering (slowing inference).
Memory Weaver — weaving memory into thought
This is MemGen's most elegant piece. Traditional RAG retrieves plaintext that the model must digest and integrate. The Weaver instead outputs machine-native latent token sequences — internal representations the model can directly "sense." If RAG hands you a dictionary to look things up, the Weaver is your brain directly surfacing a concept. The Weaver fuses current reasoning features + historical experience + an optional external knowledge base to generate latent memory tokens that are seamlessly woven into the main model's context — not appended, but knitted into the current reasoning thread.
Technical Details: LoRA Only, Main Model Untouched
You can attach MemGen to an already-trained LLM without retraining the whole model. Iteration cost is minimal and experiment cycles are short.
Three Memory Types Emerge Spontaneously
The most surprising finding: without any manual design, the system spontaneously differentiated three memory types mirroring human cognition:
| Memory Type | Function | Human Analogue | |---|---|---| | Planning Memory | Task decomposition strategies, goal hierarchies | Long-term procedural memory | | Procedural Memory | Concrete operation steps, tool-call patterns | Muscle / skill memory | | Working Memory | Temporary state tracking for the current task | Working memory / attention buffer |
This differentiation arose on its own during training — the model learned which "drawer" each experience belongs in.
Benchmarks: Sweeping Eight Leaderboards
MemGen was tested comprehensively on eight authoritative benchmarks, including ALFWorld, GSM8K, and GPQA.
| Compared Against | Improvement | |---|---| | Mainstream retrieval-memory methods (ExpeL, AWM, etc.) | Up to +38.22% | | GRPO-optimized baselines | +13.44% | | Inference time | Compressed by up to 75.4% |
Cross-domain stability: learning math doesn't erase science
After training on math tasks, scores on scientific reasoning and code tasks rose simultaneously. No catastrophic forgetting — instead, positive transfer. Because MemGen doesn't touch the main model's weights, math-related latent memories don't overwrite science and code knowledge; when a science problem appears, the Trigger唤起 — evokes — the appropriate latent memories rather than forcing math-style thinking onto it.
The three-way memory differentiation also emerged without any human annotation.
Why This Is a Paradigm Shift in Memory Architecture
MemGen marks a turn from "store-and-retrieve" to "generate-and-weave." The traditional metaphor is a library: books (knowledge), a catalog (index), borrowing when needed. MemGen's metaphor is the brain: experience isn't stored somewhere waiting to be read — it is regenerated on demand and rewoven into current thought.
Three deep implications:
1. Memory is no longer static. A RAG knowledge base is dead unless manually updated. MemGen's latent memory is alive — each invocation can re-weave it according to the current context, producing new combinations and insights.
2. No context fragmentation. Retrieved RAG text consumes precious context window. MemGen's latent memory is a compact internal representation — far fewer tokens for the same information (which explains the 75.4% inference-time reduction).
3. True modularity. The main model is the main model; memory is memory. You can swap memory modules (math, coding, medical) like a SIM card.
One-Line Summary
> MemGen demonstrates that AI memory need not be "text stored externally" nor "knowledge burned into weights." It can be implicit experience generated in real time during reasoning — surfacing naturally like the human subconscious, never interrupting thought, never polluting the model, yet significantly boosting reasoning ability.
When AI finally has an "unconscious," it takes another step toward true intelligence.
---
Paper Information
*This article is also published at zhichai.net*