Paper Overview
Field: NLP Authors: Yiming Zhang, Jinghong Zhang, Haoran Zhao, Yiren Ma, Chunlei Zhao Published: 2026-09-18 arXiv: 2609.22043
Summary
Memory systems for large language models have focused predominantly on efficient retrieval, whereas the decision of whether retrieved memories should be trusted has received comparatively little attention. When the memory store contains conflicting positions, standard retrieval-augmented generation (RAG) blindly injects memories and amplifies hallucinations: in models susceptible to memory injection, the RAG hallucination rate under conflicting memories is markedly higher than that of a memory-free baseline.
Inspired by memory signaling mechanisms in the prefrontal cortex, the authors propose the Memory Decision Layer (MDL), a zero-parameter memory decision controller situated between the retrieval and generation stages. Its core is a three-signal complementary encoder that fuses relevance, reliability, and task risk via QR-based orthogonal subspace projection, combined with a meta working-memory signal to form an interpretable decision representation that quantifies the trustworthiness of retrieved memories. MDL explicitly decouples confidence from consistency and introduces risk inversion and explicit abstention.
Key Results
- Reduces hallucination rates under conflicting memories by ~56.04% in general scenarios across mainstream LLMs and multiple open-source datasets
- Approaches zero hallucination in high-risk scenarios
- Fully white-box: pure geometric operations, no trainable parameters
- Adds only ~0.14 ms per decision — roughly 50x faster than the preceding embedding retrieval step, and four to five orders of magnitude faster than LLM self-evaluation calls
Original Abstract
> Memory systems for large language models have focused predominantly on efficient retrieval, whereas the decision of whether retrieved memories should be trusted has received comparatively little attention. When the memory store contains conflicting positions, standard retrieval-augmented generation (RAG) blindly injects memories and amplifies hallucinations: in models susceptible to memory injection, the RAG hallucination rate under conflicting memories is markedly higher than that of a memory-free baseline. Inspired by memory signaling mechanisms in the prefrontal cortex, we propose the Memory Decision Layer (MDL), a zero-parameter memory decision controller situated between the retrieval and generation stages. Its core is a three-signal complementary encoder that fuses relevance, reliabi...
---
*Auto-collected on 2026-09-22.*