English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

CodeQuant2: Clustering-Based LUT Quantization for Large Reasoning Models

Forum topic · 小凯 · 2026-11-09

Summary

CodeQuant2 (arXiv:2611.09868) is a clustering-based lookup-table (LUT) quantization framework designed for low-precision large reasoning models (LRMs). LRMs generate long intermediate reasoning traces that raise inference costs, but aggressive quantization often degrades reasoning accuracy. The authors observe larger activation fluctuations during the first decoding steps of reasoning, meaning calibration that omits these early states can miss critical activation dynamics. CodeQuant2 addresses these challenges by mitigating activation outliers with rotation and scaling, using clustering to represent non-uniform weight distributions, and adaptively adjusting codebook sizes according to per-layer quantization difficulty. It also improves calibration sample selection to better match actual decoding statistics observed during inference. Combined with a custom LUT CUDA kernel, CodeQuant2 delivers significant accuracy gains on reasoning benchmarks and achieves up to 2.85x speedup over BF16 inference.

论文概要

Research Area: NLP Authors: Xingyu Liu, Xiangyang Yin, Tianhua Xia, Haiyu Wang, Sai Qian Zhang Published: 2026-11-06 arXiv: 2611.09868

Abstract

Large reasoning models (LRMs) improve task performance by generating longer intermediate reasoning traces, but this substantially increases inference costs and makes low-precision quantization increasingly important. However, aggressively quantized models often perform poorly on reasoning tasks.

The authors observe larger activation fluctuations during the first decoding steps of reasoning. Calibration that omits these early states may miss their activation fluctuations. To address these reasoning-specific quantization challenges, the authors propose CodeQuant2, a clustering-based LUT quantization framework for low-precision large reasoning models.

Key techniques:

  • Mitigates activation outliers with rotation and scaling
  • Uses clustering to represent non-uniform weight distributions
  • Adapts codebook sizes to match per-layer quantization difficulty
  • Improves calibration sample selection to better match real decoding statistics observed at inference time
Combined with a custom LUT CUDA kernel, CodeQuant2 achieves significant accuracy improvements on reasoning benchmarks while delivering up to 2.85x speedup over BF16.

*Auto-collected on 2026-11-10.*

Tags

#codequant2#quantization#large-reasoning-models#lut-quantization#inference-acceleration#nlp#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178636982