[论文] ReasonAlloc: Hierarchical Decoding-Time KV Cache Budget Allocation for...

论文概要

研究领域: ML
作者: Wenhao Liu, Hao Shi, Yunhe Li, Weizhi Fei, Xiangyuan Wang, Mengzhe Ruan, Hanxu Hou, Peisong Wang, Linqi Song, Shuang Qiu
发布时间: 2026-06-09
arXiv: 2606.11164

中文摘要

长链推理（CoT）导致KV缓存快速增长，造成推理瓶颈。现有解码时压缩方法通常假设预算在层和头之间均匀分布。ReasonAlloc将KV压缩重新表述为层次化预算分配问题：离线层wise预分配策略捕获架构驱动的需求模式（"推理波"），在线头wise策略在解码时基于实时效用将资源重新分配给信息丰富的头。在MATH-500、AIME 2024上，使用DeepSeek-R1-Distill和AceReason模型，在小预算（128-512 token）下取得最大增益。

原文摘要

Long chain-of-thought (CoT) trajectories in large language model (LLM) reasoning cause severe inference bottlenecks due to rapid key-value (KV) cache growth. Current decoding-time compression methods mitigate this issue via token eviction, but typically assume a uniform budget distribution across all layers and heads. In contrast, existing non-uniform budget allocation methods are predominantly designed for the static prompt prefill phase, and they do not capture the stepwise context demands of autoregressive reasoning. To bridge this gap, we propose ReasonAlloc, a training-free framework that recasts decoding-time KV compression as a hierarchical budget allocation problem. ReasonAlloc operates at two complementary levels: an offline layer-wise preallocation strategy captures an architectu...

自动采集于 2026-06-11

#论文 #arXiv #ML #小凯