MemCon: Memory Is Not a Warehouse, It's a Controlled Process for LLM Agents

An Overlooked Problem

Equipping LLM agents with memory systems has become standard practice. Voyager uses a skill library, MemoryBank uses reflective summaries, OAgents uses rule induction—solutions are flourishing. But these systems share a common assumption: the way memory is accessed is fixed.

Specifically, they all use "hand-designed heuristics" to decide when to retrieve, what to retrieve, and how much. The most common pattern: perform a nearest-neighbor retrieval every turn and stuff the top-k results into the context.

In the paper *Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents*, UCLA's Eric Hanchen Jiang, Zhi Zhang, and colleagues argue: this assumption is wrong. Optimal memory behavior is context-dependent—no single fixed rule works for all situations.

Four Counter-Intuitive Scenarios

The paper lists four typical situations, each requiring a different memory strategy:

1. Early in a task: The memory store is still sparse. Retrieval at this point yields irrelevant fragments—optimal strategy is to retrieve rarely or not at all. 2. Repeated task types: If a similar task was solved before, rather than doing another fuzzy retrieval, the optimal move is to directly reuse the previous plan (plan injection). 3. The agent is stuck: If the current retrieval didn't help, the optimal strategy isn't giving up but re-retrieving with alternative queries. 4. Long task streams: The memory store grows and retrieval quality degrades. The optimal strategy is periodic consolidation and pruning, and even active forgetting.

These four situations call for four completely different actions. Fixed heuristics can only force one strategy onto all scenarios and will inevitably fail in some of them.

How MemCon Works

MemCon (Memory as a Controlled Process) models memory operations as a Markov Decision Process (MDP):

• State \(\mathcal{S}\): Contextual features of the current task (task stage, memory size, whether stuck, etc.)
• Actions \(\mathcal{A}\): {Retrieve (at different depths), PlanInject, Re-Retrieve, Consolidate, Forget, NoOp}—9 actions total
• Reward \(\mathcal{R}\): Task success (+1) or failure (-0.5), plus an efficiency reward
• Transition \(\mathcal{T}\): Determined by the environment and memory state

An online policy is then learned to decide which action to take in each state.

A key design choice: no neural network for the policy—instead, a tabular contextual bandit with UCB exploration.

Why?

• No pretraining needed: learns from scratch via task feedback
• No extra LLM calls: the policy itself is lightweight and adds no inference cost
• Fast convergence: converges within a few dozen tasks (the paper proves a regret bound of \(O(\log T / T)\))
• Interpretable: every state-action pair's Q value can be inspected

The policy update uses backward-discounted cumulative reward: \(\gamma^{|\text{ep}|-j-1} r_i\), giving later actions in a task greater weight (they are closer to the final outcome).

An Analogy

Think of an LLM agent's memory system as a librarian.

The traditional approach: no matter who asks, follow the same procedure—search the catalog, fetch the 5 most relevant books, hand them over. This works for some readers but wastes effort for many.

The librarian MemCon trains adapts to each reader:

• First-time visitor (early task): "Browse around freely, no catalog needed."
• Returning reader on a familiar topic (repeated task): "Here's the stack you borrowed last time—just take it."
• Reader saying "I can't find what I want" (stuck): "Let me search again with different keywords."
• Overflowing shelves (long task stream): "Let me archive a few old books to make room."

This librarian doesn't need a PhD—it just needs to learn what to do in which situation. That's exactly what a contextual bandit is good at.

Experimental Results

Tested on 6 benchmarks (ALFWorld, ScienceWorld, Blocksworld, TriviaQA, WebWalkerQA, GAIA), 3 agent frameworks, and 3 LLM backbones (Claude Sonnet-4, DeepSeek-V3.2, Qwen):

• Task success rate improved by up to 15.2 percentage points
• Token consumption reduced by 5–20% (by eliminating unnecessary retrievals)
• Consistently outperformed baselines including Voyager, MemoryBank, Generative Memory, OAgents, and G-Memory across all benchmarks

Notably, the token efficiency gain means MemCon is not only more accurate but also cheaper—because it learned "when *not* to retrieve."

Why This Matters

For agent design: MemCon reveals a neglected dimension—the control policy of a memory system is itself something that must be learned. Prior work optimized what to store and how to retrieve, but ignored decisions like when to retrieve, how much, and when to forget.

For RL in agents: MemCon uses the simplest contextual bandit, not deep reinforcement learning. This shows that for many agent subproblems, lightweight RL suffices—no need for heavy machinery like PPO or DQN. The key is getting the problem formulation right, not algorithmic complexity.

For sustainable agent operation: Memory bloat during long task streams is a pain point for all agent systems. MemCon's consolidation and forget actions offer an automated solution—no manual cleanup needed; the policy learns when to slim down on its own.

Honest Limitations

• Hand-crafted state design: although the policy is learned, the state features \(\phi\) are manually designed. Different tasks may need different features.
• Binary feedback: reward comes only from task success/failure—not rich enough for open-ended tasks without clear success criteria.
• Fixed action space: 9 predefined actions. If the optimal policy requires operations outside this space, MemCon can't discover them.
• Limited tabular state space: with high-dimensional states, the table becomes sparse. The paper uses discretised state keys to mitigate this, at the cost of some information loss.

Conclusion

MemCon's core insight isn't an algorithmic innovation but a perspective shift: memory is not a static warehouse but a dynamic, controlled process.

This echoes cognitive science's "memory control" theory—human memory access is not a fixed pipeline but a process dynamically regulated by the prefrontal cortex according to current goals. We sometimes actively recall (retrieve), sometimes rely on habit (plan inject), sometimes try a different cue (re-retrieve), sometimes organize old memories (consolidate), and sometimes deliberately forget (forget).

LLM agents are finally learning to do this too.

---

Paper: Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents Authors: Eric Hanchen Jiang, Zhi Zhang, Yuchen Wu, Ying Nian Wu (UCLA) Code: https://github.com/ericjiang18/MemCon