> Source: An Interpretable Memory Decision Controller for LLM Agents Based on Three-Signal Complementarity, arXiv:2609.22043 > Authors: Yiming Zhang, Jinghong Zhang, Haoran Zhao, Yiren Ma, Chunlei Zhao (Tianjin University of Technology)
Introduction: An Outdated Pharmacopoeia
Ask a medical agent: "Can a pregnant woman take drug X?" It retrieves a highly relevant record from memory—a 1998 pharmacopoeia marked "safe"—and injects it into context, answering "safe." What the system doesn't know: the pharmacopoeia was revised in 2015 to explicitly prohibit use during pregnancy. The retriever only answers "which memory is most relevant," never "is this memory trustworthy."
This is the core failure mode of RAG: retrieve-and-adopt. Every retrieved memory is injected unconditionally, with no trust assessment. The paper's experiment shows that when the memory store contains both correct answers and common misconceptions, RAG's hallucination rate reaches 53.0%—more than double the no-memory baseline (23.0%). Adding memory makes things worse.
The paper proposes a Memory Decision Layer (MDL) between retrieval and generation to judge whether each retrieved memory deserves trust.
Core Design: Three-Signal Complementarity + Zero-Parameter Geometry
1. Inspiration: The Prefrontal Cortex
The authors draw on prefrontal memory-signal mechanisms. Decisions integrate three signal types:
- Relevance: how related is the memory to the query?
- Reliability: how trustworthy is the memory itself? Authoritative source? Outdated?
- Task Risk: how sensitive is the question? How costly is an error?
- Interpretable: outputs C (consistency) and α (confidence) are audit interfaces showing exactly why a memory is trusted or not
- Fast: 40.1 µs per decision (~0.14 ms including lexical signal aggregation), ~50x faster than embedding retrieval, and 4–5 orders of magnitude faster than LLM self-evaluation
- White-box: no black-box networks; all operations have closed-form solutions
- Confidence: how certain is the model of its answer
- Consistency: whether the memory aligns with task direction
All three matter. A highly relevant but outdated pharmacopoeia might be tolerable in low-risk settings, but must be rejected in medical ones.
2. QR Orthogonal Subspace Projection
The three scalar signals are mapped into three orthogonal subspaces of a 16-dimensional space (dimension allocation 5+5+6) via QR decomposition, keeping them geometrically independent. A "meta-working-memory" signal provides global context.
The design is fully zero-parameter: no trained weights, only geometric operations (QR decomposition, vector norms, cosine angles). Benefits:
3. Decoupling Confidence from Consistency
Standard RAG conflates relevance and trustworthiness—high retrieval score means trusted. MDL explicitly decouples:
4. Four-Level Action Decisions
Rather than a binary use/don't-use choice, MDL outputs four actions:
1. Adopt: memory is trustworthy; inject into context 2. Correct: partially trustworthy; correct then inject 3. Abstain: untrustworthy, but generation is not blocked 4. Reject: harmful; trigger explicit refusal
Experimental Results: Zero Hallucination in High-Risk Scenarios
1. Main Experiment: TruthfulQA
With conflicting memories in the store:
| System | Overall hallucination | High-risk hallucination | |--------|----------------------|-------------------------| | No-memory baseline | 23.0% | 12.4% | | Standard RAG | 53.0% | 63.0% | | MDL | 23.3% | 0.0% |
In high-risk scenarios (medical, legal, financial), MDL drives hallucination from 63.0% to 0.0%. Risk inversion triggers abstention on high-risk queries (A ≥ 0.85), outright refusing unreliable memories. Overall hallucination drops 56.04% (53.0% → 23.3%).
2. Cross-Model Generalization
Validated on three LLMs: gemma-4-E4B-it, deepseek-v4-flash, gemini-3-flash-preview. MDL works on all, though gains vary with model capability—stronger reasoners (deepseek-v4-flash) are more resistant to memory injection themselves, so MDL's benefit is smaller; weaker models benefit more.
3. Cross-Dataset Generalization
On HaluEval, MDL also works: overall hallucination drops from 6.5% (RAG) to 3.5%; high-risk from 2.7% to 1.3%.
4. Latency Comparison
| Component | Latency | |-----------|---------| | Embedding retrieval | ~7 ms | | MDL decision | 0.14 ms | | LLM self-evaluation | 200–500 ms |
MDL's overhead is negligible—50x faster than retrieval, thousands of times faster than asking an LLM whether its own memory is trustworthy.
Why This Matters
1. The "retrieve-and-adopt" default is wrong
Nearly all RAG systems assume retrieved memories should be used. This paper proves the assumption wrong: with conflicting memories, RAG worsens hallucination from 23% to 53%. This mirrors the "proxy goal trap": retrieval relevance is the proxy; memory trustworthiness is the real goal. Optimizing retrieval precision doesn't optimize memory trustworthiness.
2. Revival of zero-parameter white-box methods
MDL uses no trained weights—pure geometric operations—and outperforms LLM self-evaluation. This follows the same path as DoM (Difference of Means) reward-hacking detection: white-box revival. Linear algebra tools (QR decomposition, orthogonal projection, cosine angles) may be more reliable for trust judgments than a black-box LLM reflecting on itself—especially since LLM self-evaluation can itself be compromised by memory injection, creating circular dependence.
3. Another case of "judge-gate decoupling"
MDL's core contribution is decoupling retrieval from adoption—retrieval finds memories; an independent decision layer judges them. This matches the same design pattern as Chronicle's guard, ODA's recall head, and PagedWeight's bit-width floor: add an independent gate on the critical path rather than trusting upstream.
4. Elegance of risk inversion
Most safety mechanisms are linearly "more risk, more caution." MDL's risk inversion is more aggressive—beyond a threshold, the signal direction flips, turning "adopt" into "reject." Not caution, but structural refusal. In medical, legal, and financial settings where errors are costly, abstention beats using unreliable memory.
Limitations and Honest Assessment
1. The memory store is controlled: conflicting memories are artificially constructed; real-world conflicts may be more complex and subtle 2. Task risk is rule-assigned: TruthfulQA uses category mapping; HaluEval uses keyword matching ({0.20, 0.50, 0.85}). Real-world risk assessment may need more dynamic methods 3. No end-to-end validation on real agent systems: experiments are single-turn QA, not multi-turn agent tasks, where cumulative memory conflicts could be more complex
But the core claim—that there should be an independent trust decision layer between retrieval and adoption—stands. That slot used to be empty; now it's filled.
Conclusion
Years into the RAG story, mainstream work optimizes "how to retrieve more accurately." This paper highlights an overlooked problem: retrieving accurately doesn't mean using correctly.
MDL's value isn't complexity (it's minimal)—it's making explicit a step everyone defaulted past: "is this memory trustworthy?" At 0.14 ms per decision, it buys a drop from 63% to 0% hallucination in high-risk scenarios.
Sometimes the most important innovation isn't "doing better"—it's filling in the step everyone else skipped.
Paper: https://arxiv.org/abs/2609.22043 HTML version: https://arxiv.org/html/2609.22043v1