[论文] The Illusion of Equivalency: Statistical Characterization of Quantizat...

论文概要

研究领域: ML
作者: Baha Rababah, Cuneyt Gurcan Akcora, Carson K. Leung
发布时间: 2025-07-12
arXiv: 2507.08705

中文摘要

训练后量化被广泛用于在资源受限环境中部署大语言模型，但其评估几乎完全依赖于准确率和困惑度。我们表明这些指标无法捕捉量化引起的行为变化。我们引入了正确性一致性，一个决策级指标，衡量基础模型与其量化变体在正确预测上的重叠，独立于绝对准确率。在多个模型和从8位到2位的量化方案中，我们发现即使在任务性能看似保持的情况下，中等量化下也会出现行为分歧。为解释这一效应，我们将量化分析为对注意力权重的结构性操作，并使用统计和分布度量来量化逐层失真。我们的结果揭示了低位宽下的非线性断点，并表明查询和键投影始终比值和输出投影更敏感。这些发现暴露了基础模型与量化模型之间的等价性幻觉，并推动超越传统性能指标的行为评估。

原文摘要

Post-training quantization is widely used to deploy large language models in resource-constrained settings, yet its evaluation relies almost exclusively on accuracy and perplexity. We show that these metrics fail to capture behavioral changes induced by quantization. We introduce correctness agreement, a decision-level metric that measures overlap in correct predictions between a base model and its量化变体, independent of absolute accuracy. Across multiple models and quantization schemes from 8-bit to 2-bit, we find that behavioral divergence emerges under moderate quantization even when task performance appears preserved. To explain this effect, we analyze quantization as a structural operator on attention weights and quantify layer-wise distortions using statistical and distributional measur...

自动采集于 2026-07-13

#论文 #arXiv #ML #小凯