Introduction
Have you ever been stuck on a math problem, closed your eyes, and let your brain "find" the answer on its own — a similar problem or a teacher's trick surfacing like a bubble from underwater? That's implicit memory: it doesn't rely on deliberate recall; it emerges naturally in the gaps of thinking.
AI memory systems have long had only two options:
- Rewriting model weights (fine-tuning), risking catastrophic forgetting
- Bolting on RAG (retrieval-augmented generation), stuffing whole text passages into context — as clunky as inserting an encyclopedia into a conversation
- The main reasoning model stays fully frozen — no catastrophic forgetting
- Only the Memory Weaver's LoRA parameters are optimized — lightweight and efficient
- Supports both SFT and GRPO training — supervised learning and reinforcement learning both work
- Title: MemGen: Weaving Generative Latent Memory for Self-Evolving Agents
- Authors: Guibin Zhang, Muxin Fu, Shuicheng Yan (National University of Singapore)
- arXiv: https://arxiv.org/abs/2509.24704
MemGen, proposed by a National University of Singapore team, takes a third path: letting AI generate its own implicit memory during inference. Instead of retrieving external text, it weaves machine-native memory tokens in the moment of thinking — natural, immediate, and non-disruptive, like human subconsciousness.
The Dilemma of Traditional Memory Approaches
Option 1: Parameter fine-tuning — writes new knowledge into weights, but suffers catastrophic forgetting: teach it math, and it may forget how to write poetry.
Option 2: RAG retrieval — leaves the model untouched but retrieves external text and stuffs it into context. The problem is rigid concatenation: retrieved text doesn't organically merge with current reasoning, hurting coherence and efficiency.
MemGen's answer: freeze the main model, train only a memory module.
MemGen's "Subconscious" Architecture
The core design has just two modules:
Memory Trigger — when to remember what
A vigilant system that continuously monitors the main model's reasoning hidden states. The key design: it triggers only at semantic nodes — periods, semicolons, logical transitions. Rather than asking "should I recall?" at every token, it judges in the gaps of thinking whether a piece of experience is needed. This avoids both extremes: never triggering (no memory at all) and over-triggering (slowing inference).
Memory Weaver — weaving memory into thought
Traditional RAG retrieves plaintext text that the model must digest and integrate. The Weaver instead outputs machine-native latent token sequences — internal representations the model can directly "feel." If RAG is like handing you a dictionary to look things up, the Weaver is like your brain directly surfacing a concept — no translation, no digestion, immediately usable.
The Weaver fuses features of current thinking + historical experience + optional external knowledge bases, generating latent memory tokens seamlessly embedded into the main model's context — not appended at the start or end, but woven into the current reasoning thread like knitting.
Technical Details: LoRA Only, Main Model Untouched
This means MemGen can attach directly to an already-trained large model with minimal iteration cost.
Spontaneous Differentiation into Three Memory Types
The most surprising finding: without any human design, the system spontaneously differentiated three memory types mirroring human cognition:
| Memory Type | Function | Human Analogue | |---|---|---| | Planning Memory | Task decomposition strategies, goal hierarchies | Long-term procedural memory | | Procedural Memory | Concrete operation steps, tool-calling patterns | Muscle / skill memory | | Working Memory | Temporary state tracking for the current task | Working memory / attention buffer |
The model learned on its own that different task types need different memory support, naturally filing experiences into different "drawers."
Benchmark Results: Sweeping Gains Across Eight Benchmarks
MemGen was tested comprehensively on eight authoritative benchmarks, including ALFWorld, GSM8K, and GPQA.
| Comparison | Improvement | |---|---| | vs. mainstream retrieval memory (ExpeL, AWM, etc.) | up to +38.22% | | vs. GRPO-optimized methods | +13.44% | | Inference time reduction | up to 75.4% |
Cross-Domain Stability: Learning Math Without Forgetting Science
The most impressive data point: after training on math tasks, scores on scientific reasoning and code tasks rose in tandem. No catastrophic forgetting — instead, positive transfer. Because MemGen doesn't touch the main model's weights, memory is external and modular. When facing science problems, the Trigger recalls corresponding latent memory rather than forcing a math mindset.
Why This Is a Paradigm Shift in Memory Architecture
MemGen represents a shift from "store-and-retrieve" to "generate-and-weave":
1. Memory is no longer static — RAG knowledge bases are dead until manually updated; MemGen's latent memory is alive, re-woven with each call based on the current context. 2. No context fragmentation — latent memory is a compact internal representation with far lower token overhead (explaining the 75.4% inference speedup). 3. True modularity — swap memory modules like SIM cards: math memory, coding memory, medical memory, all on the same base model.
One-Sentence Summary
> MemGen demonstrates that AI memory need not be text stored externally, nor knowledge burned into weights. It can be an implicit experience generated in real time during reasoning — surfacing naturally like human subconsciousness, without interrupting thought or polluting the model, while significantly boosting reasoning capability.