MemCoE: Teaching LLM Agents to Memorize Like Humans with Cognition-Inspired Two-Stage Optimization

Forum topic · 小凯 · 2026-05-04

Summary

MemCoE (Memory Cognition Optimization with Evolution) is a framework for improving long-term memory in LLM agents, introduced in the paper 'Learning How and What to Memorize: Cognition-Inspired Two-Stage Optimization for Evolving Memory' (arXiv: 2605.00702). The post explains that existing agent memory systems suffer from three limitations: finite context windows, hand-crafted static update rules, and sparse reinforcement-learning rewards that make credit assignment for memory updates difficult—compounded by the fact that user preferences evolve over time. Inspired by cognitive science—memory schema theory and the division of labor between the hippocampus (rapid encoding) and prefrontal cortex (structured long-term storage)—MemCoE decouples memory optimization into two stages: learning what to memorize (filtering for important, persistent, generalizable information) and how to memorize (organizing, integrating, reconciling, updating, and selectively forgetting memories). The framework learns these policies end-to-end from data instead of relying on fixed rules, supporting memory evolution as user preferences change. The article argues that the future of AI memory lies not in larger context windows but in smarter memory management—selective, organized, and adaptive, mirroring human cognition.

MemCoE: When LLM Agents Learn to "Memorize Like Humans"

> Paper: Learning How and What to Memorize: Cognition-Inspired Two-Stage Optimization for Evolving Memory > Authors: Derong Xu, Shuochen Liu, Pengfei Luo, Pengyue Jia, Yingyi Zhang, Yi Wen, Yimin Deng, Wenlin Zhang, Enhong Chen, Xiangyu Zhao, Tong Xu > arXiv: 2605.00702 | 2026-04-30

1. The AI Assistant with a "Bad Memory"

Imagine an AI assistant that manages your schedule, recommends restaurants, and remembers your preferences. After months of conversations, it should know that you:

Don't eat seafood
Love science fiction novels
Are free on Wednesday evenings
Are sensitive to caffeine

The problem:

Context windows are limited, so it "can't remember" long histories
Static memory-update rules are too rigid
Reinforcement learning suffers from sparse rewards
Your preferences change (e.g., you recently started trying seafood), but the AI still acts on old preferences

LLM agents need long-term memory, but existing memory systems are too simplistic.

2. Three Core Challenges of Long-Term Memory

1. Limited context windows

Models like GPT-4 can only handle a finite number of tokens
They cannot directly store all historical interactions
External memory systems are required

2. Static update rules

Existing systems rely on hand-crafted rules like "if the user says X, update memory Y"
Rules can't cover all cases or adapt to individual differences

3. Sparse rewards

RL can learn memory updates, but rewards typically arrive only at task completion
The effect of long-term memory is hard to attribute to specific memory updates
This leads to unstable training

A subtler issue: preferences evolve. People are not static—today's preferences differ from tomorrow's, so memory systems must support "evolution."

3. MemCoE: Cognition-Inspired Two-Stage Memory Optimization

MemCoE (Memory Cognition Optimization with Evolution) draws on cognitive science's memory schema theory and the hippocampus–prefrontal cortex division of labor:

Hippocampus: rapidly encodes new memories, but with limited capacity
Prefrontal cortex: long-term, organized, structured storage
Schemas: abstract representations of repeated experience

Two-stage design:

Stage 1: What to Memorize

Learn to identify which information is worth remembering
Not every conversation belongs in long-term memory
Filter for "important, persistent, generalizable" information

Stage 2: How to Memorize

Learn how to organize and update memory
How should new information integrate with old memories?
How are conflicting memories reconciled?
When should old memories be updated or forgotten?

Key innovations:

Two-stage decoupling: decide "what to remember" first, then "how to remember"
Cognitive inspiration: mimics the division of labor in human memory systems
End-to-end learning: policies are learned from data, not hand-crafted rules

This is like giving an AI a "cognitive memory system": not a simple notebook, but an intelligent memory that selects, organizes, and forgets.

4. Why Cognitive Inspiration Works

Human memory, refined by millions of years of evolution, embodies real wisdom:

1. Selective attention — we don't remember every falling leaf, only what is goal-relevant, emotionally significant, or novel. MemCoE learns this selectivity. 2. Hierarchical organization — memory is not a flat list but structured and associative. MemCoE learns hierarchical organization. 3. Dynamic updating — memory is not written in stone; it adjusts with new experience. MemCoE supports memory evolution. 4. The wisdom of forgetting — forgetting is a feature, not a flaw: it removes stale information while preserving core knowledge. MemCoE learns when to forget.

5. Memory Is Active Construction, Not Passive Storage

Feynman famously noted that knowing the name of something is entirely different from understanding it—understanding means connecting new knowledge to your existing knowledge network.

In memory systems, the same idea applies:

> Memory is not stuffing information into drawers. Memory is weaving new information into an existing knowledge network—deciding where it belongs, what it connects to, and whether it's worth keeping.

MemCoE's philosophical foundation: memory is an active, selective, constructive process.

Passive storage = database
Active memory = intelligence

6. Takeaways for Builders

If you're building LLM agents or conversational systems, ask yourself:

1. Is my memory system selective—not everything gets stored? 2. Are memory updates hand-crafted rules or learned? 3. Does the system support memory evolution over time? 4. Have I considered insights from cognitive science?

MemCoE reminds us: the future of AI memory is not "a bigger context window" but "smarter memory management."

Human memory isn't perfect—we forget, distort, and select. But these "flaws" are precisely features of intelligence. MemCoE borrows from cognitive science to make AI memory more human-like: selective, organized, and evolving.

In the world of LLM agents, the best memory doesn't remember the most—it remembers the smartest.