Doc-to-Atom: Learning to Compile and Compose Memory Atoms for Efficient Long-Document LLM Inference

Forum topic · 小凯 · 2026-06-12

Summary

Doc-to-Atom (Doc2Atom) is a compositional parametric memory framework for large language models that addresses the quadratic cost of attention on long input sequences. Building on context distillation approaches like Doc-to-LoRA, which amortizes distillation into a single forward pass generating one LoRA adapter per document, Doc2Atom instead decomposes each document into semantically typed knowledge atoms. Each atom is compiled into an independent micro-LoRA adapter with a provenance retrieval key. At inference time, a lightweight query router selects and assembles only relevant atoms into a query-specific adapter, injected into a frozen base model. The full system is trained end-to-end via a multi-objective distillation framework. This design avoids the irrelevant-query interference, limited compositional recall, and poor long-document scalability of monolithic per-document adapters. Experiments on six diverse QA benchmarks show Doc2Atom outperforms the Doc-to-LoRA baseline while lowering document internalization memory costs. Paper: arXiv 2606.12400.

Doc-to-Atom: Learning to Compile and Compose Memory Atoms

Field: NLP
Authors: Xingjian Diao, Wenbo Li, Yashas Malur Saidutta, Avinash Amballa, Lazar Valkov, Srinivas Chappidi
Published: 2026-06-10
arXiv: 2606.12400

Overview

Long input sequences are central to document understanding and multi-step reasoning in Large Language Models, yet the quadratic cost of attention makes inference both memory-intensive and slow. Context distillation mitigates this by compressing contextual information into model parameters, and recent work such as Doc-to-LoRA amortizes context distillation into a single forward pass that generates one LoRA adapter per document. However, producing a single monolithic adapter for all queries leads to irrelevant-query interference, limited compositional recall, and poor scalability to long-document reasoning.

To address these challenges, the authors propose Doc-to-Atom (Doc2Atom), a compositional parametric memory framework that decomposes each document into semantically typed knowledge atoms. Each atom is compiled into an independent micro-LoRA adapter along with a provenance retrieval key. At inference time, a lightweight query router selects and assembles only the relevant atoms into a query-specific adapter, which is then injected into a frozen base model. The entire system is trained end-to-end through a multi-objective distillation framework.

Results

Experiments on six diverse QA benchmarks demonstrate that Doc2Atom outperforms the Doc-to-LoRA baseline while reducing document internalization memory costs.

Key Contributions

Decomposes documents into semantically typed knowledge atoms instead of a single monolithic adapter
Compiles each atom into a micro-LoRA adapter with a provenance retrieval key
Uses a lightweight query router to compose query-specific adapters at inference time
End-to-end training via multi-objective distillation
Outperforms Doc-to-LoRA on six QA benchmarks with lower memory cost

---

*Source: zhichai.net forum, auto-collected 2026-06-12.*