[论文] CodeQuant2: Enhanced Outlier Mitigation and Adaptive Clustering for Lo...

论文概要

研究领域: NLP 作者: Xingyu Liu, Xiangyang Yin, Tianhua Xia, Haiyu Wang, Sai Qian Zhang 发布时间: 2026-11-06 arXiv: 2611.09868

中文摘要

大型推理模型（LRM）通过生成更长的中间推理轨迹提升任务性能，但这显著增加了推理成本，使低精度量化越来越重要。然而激进量化的模型在推理任务上往往表现不佳。作者观察到推理过程中初始解码步骤的激活波动更大，省略这些早期状态的校准可能错过其激活波动。为应对这些推理特有的量化挑战，提出 CodeQuant2——基于聚类的LUT量化框架。CodeQuant2 通过旋转和缩放缓解激活离群点，用聚类表示非均匀权重分布，并自适应调整码本大小以适应每层的量化难度。基于对推理时激活行为的观察，CodeQuant2 进一步改进校准样本选择以更好匹配实际解码统计。结合定制的LUT CUDA内核，CodeQuant2 在推理基准上带来显著精度提升，同时实现最高 2.85 倍于 BF16 的加速。

原文摘要

Large reasoning models (LRMs) improve task performance by generating longer intermediate reasoning traces, but this substantially increases inference costs and makes low-precision quantization increasingly important. However, aggressively quantized models often perform poorly on reasoning tasks. We observe larger activation fluctuations during the first decoding steps of reasoning. Calibration that omits these early states may miss their activation fluctuations. To address these reasoning-specific quantization challenges, we propose \textbf{CodeQuant2}, a clustering-based LUT quantization framework for low-precision large reasoning models. CodeQuant2 mitigates activation outliers with rotation and scaling, uses clustering to represent non-uniform weight distributions, and adapts codebook s...

#论文 #arXiv #NLP #小凯