Summary
This paper addresses knowledge editing in language models: updating selected facts while preserving nearby but irrelevant behavior. The authors study a memory-assisted setting where an edit memory is retrieved at inference time and a parameter-efficient adapter corrects the model's object preference. Their key insight is that the central design question is not only how to write an edit but also when to suppress it. They propose a route-specialized dual-adapter editor: a relevance router first decides whether a prompt should receive the edit memory. Routed prompts use an edit adapter trained to prefer the new object; unrouted non-direct prompts use a separate locality adapter trained to preserve or restore the original-object preference. Evaluated on three 1,000-case benchmarks (CF, ZsRE, MQuAKE) with two 7B/8B base models, the method achieves the best overall probability-preference accuracy on all three benchmarks with Llama-3.1-8B-Instruct: 0.8180 on CF, 0.8946 on ZsRE, and 0.9922 on MQuAKE, with similar trends on Qwen3-8B. Ablations show the optimal relevance-memory boundary varies by dataset—lexical neural routing is safest on CF while BGE embedding routing wins on ZsRE and MQuAKE—and that gains come mainly from separating edit injection from off-route suppression rather than simply adding LoRA capacity. arXiv: 2606.14668.
Paper Overview
- Field: Machine Learning
- Author: Yining Huang
- Published: 2026-06-12
- arXiv: 2606.14668
Summary
Knowledge editing systems must update selected facts while preserving nearby but irrelevant behavior. This paper studies the problem in a memory-assisted setting where an edit memory is retrieved at inference time and a parameter-efficient adapter corrects the model's object preference. The authors argue that the central design question is not only *how* to write an edit, but also *when* to suppress it.
Method: Route-Specialized Dual Adapter Editor
- A relevance router first decides whether a prompt should receive the edit memory.
- Routed prompts use an *edit adapter* trained to prefer the new object over the original object.
- Unrouted non-direct prompts use a separate *locality adapter* trained to preserve or restore the original-object preference.
Results
Evaluated on three 1,000-case protocols (CF, ZsRE, MQuAKE) under identical memory protocols and two 7B/8B base models:
- On Llama-3.1-8B-Instruct, the method achieves the best overall probability-preference accuracy on all three benchmarks: 0.8180 on CF, 0.8946 on ZsRE, and 0.9922 on MQuAKE.
- The same trend holds on Qwen3-8B.
Ablation Findings
- Router ablations show the relevant-memory boundary varies by dataset: a lexical neural router is safest on CF, while BGE embedding routing performs better on ZsRE and MQuAKE.
- Component and module ablations show the gains come mainly from separating edit injection from off-route suppression, rather than simply adding LoRA capacity.
---
*Auto-collected on 2026-06-16*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177981393