Context-Driven Incremental Compression (C-DIC) for Multi-Turn Dialogue: Paper Overview

Forum topic · 小凯 · 2026-06-12

Summary

This post introduces the arXiv paper 2606.12411, "Context-Driven Incremental Compression for Multi-Turn Dialogue" (C-DIC), by Yeongseo Jung et al. The paper addresses redundant attention and encoding costs that grow with dialogue length in conversational agents. Naive truncation or summarization degrades fidelity, and existing context compressors lack cross-turn memory sharing or revision, causing information loss and compounding errors. C-DIC treats a conversation as interleaved contextual threads and stores revisable per-thread compression states in a single compact dialogue memory. Each turn applies a lightweight retrieve-revise-write-back loop that shares information across turns and updates stale memory, stabilizing long-term behavior. The authors also adapt truncated backpropagation through time (TBPTT) to the multi-turn setting, learning cross-turn dependencies without backpropagating through full history. Experiments on long-form dialogue benchmarks show stable inference latency and perplexity over hundreds of turns, offering a scalable path to high-quality dialogue modeling.

Paper Overview

Field: NLP Authors: Yeongseo Jung, Jaehyeok Kim, Eunseo Jung, Jiachuan Wang, Yongqi Zhang, Ka Chun Cheung, Simon See, Lei Chen Published: 2026-06-10 arXiv: 2606.12411

English Translation of the Chinese Summary

Modern conversational agents condition on an ever-growing dialogue history at each turn, incurring redundant attention and encoding costs that grow with conversation length. Naive truncation or summarization degrades fidelity, while existing context compressors lack cross-turn memory sharing or revision, causing information loss and compounding errors in long dialogues.

The authors revisit context compression under conversational dynamics and empirically demonstrate its fragility. To improve both efficiency and robustness, they introduce Context-Driven Incremental Compression (C-DIC), which treats a conversation as interleaved contextual threads and stores revisable per-thread compression states in a single, compact dialogue memory. At each turn, a lightweight retrieve-revise-write-back loop shares information across turns and updates stale memory, stabilizing long-term behavior.

Additionally, the paper adapts truncated backpropagation through time (TBPTT) to the multi-turn setting, enabling learning of cross-turn dependencies without backpropagating through the full history.

Extensive experiments on long-form dialogue benchmarks demonstrate C-DIC's superior performance and efficiency. Notably, C-DIC maintains stable inference latency and perplexity over hundreds of dialogue turns, supporting a scalable path toward high-quality dialogue modeling.

Original Abstract (Excerpt)

> Modern conversational agents condition on an ever-growing dialogue history at each turn, incurring redundant attention and encoding costs that grow with conversation length. Naive truncation or summarization degrades fidelity, while existing context compressors lack cross-turn memory sharing or revision, causing information loss and compounding errors in long dialogues. We revisit the context compression under conversational dynamics and empirically present its fragility. To improve both efficiency and robustness, we introduce Context-Driven Incremental Compression (C-DIC), which treats a conversation as interleaved contextual threads and stores revisable per-thread compression states in a single, compact dialogue memory. At each turn, a lightweight retrieve, revise, and write-back loop sh...

--- *Auto-collected on 2026-06-12*