When to Write and When to Suppress: Route-Specialized Dual Adapters for Knowledge Editing

Paper Overview

Field: Machine Learning Author: Yining Huang Published: 2026-06-12 arXiv: 2606.14668

Summary

Knowledge editing systems must update selected facts while preserving nearby but irrelevant behavior. This paper studies the problem in a memory-assisted setting where an edit memory is retrieved at inference time and a parameter-efficient adapter corrects the model's object preference. The authors argue that the central design question is not only how to write an edit, but also when to suppress it.

Method: Route-Specialized Dual Adapter Editor

• A relevance router first decides whether a prompt should receive an edit memory.
• Routed prompts use an *edit adapter* trained to prefer the new object over the original object.
• Unrouted non-direct prompts use a separate *locality adapter* trained to preserve or restore the original-object preference.

Results

Evaluated on three 1,000-case protocols — CF, ZsRE, and MQuAKE — under the same memory protocol and two 7B/8B base models:

| Benchmark | Llama-3.1-8B-Instruct probability-preference accuracy | |---|---| | CF | 0.8180 | | ZsRE | 0.8946 | | MQuAKE | 0.9922 |

The method achieves the best overall accuracy on all three benchmarks with Llama-3.1-8B-Instruct, and the same trend holds on Qwen3-8B.

Ablation Findings

• Router ablation: the boundary of relevant memory differs across datasets — a lexical neural router is safest on CF, while BGE-embedding routing performs better on ZsRE and MQuAKE.
• Component and module ablation: the gains mainly come from separating edit injection from off-route suppression, rather than from simply adding LoRA capacity.

Original Abstract (excerpt)

> Knowledge editing systems must update selected facts while preserving nearby but irrelevant behavior. This paper studies this problem in a memory-assisted setting where an edit memory is retrieved at inference time and a parameter-efficient adapter corrects the model's object preference. We argue that the central design question is not only how to write an edit, but also when to suppress it. We introduce a route-specialized dual-adapter editor. A relevance router first decides whether a prompt should receive an edit memory...

--- *Auto-collected on 2026-06-16*