Summary
SliderQuant is a new post-training quantization (PTQ) framework for large language models (LLMs), introduced in an arXiv paper (2603.25284) by Shigeng Wang, Chao Li, Yangyuxuan Kang, Jiawei Fan, Zhonghong Ou, and colleagues. The work addresses a largely overlooked aspect of sequential PTQ pipelines: mainstream frameworks treat all layers uniformly, which can be suboptimal under challenging bit-width settings. Through empirical study of quantization impact across layers, the authors observe that shallow and deep layers are generally more sensitive to quantization than middle layers, and within those regions the very first and last layers are the most sensitive, exhibiting markedly larger quantization errors than other layers. Building on these findings, SliderQuant applies a simple adaptive sliding quantization concept to handle layers according to their sensitivity. Extensive experiments across various LLMs show that SliderQuant outperforms existing PTQ methods in both weight-only quantization and weight-activation quantization settings, offering more accurate low-bit LLM deployment.
Paper Overview
Field: Machine Learning
Authors: Shigeng Wang, Chao Li, Yangyuxuan Kang, Jiawei Fan, Zhonghong Ou, et al.
Published: 2026-03-26
arXiv: 2603.25284
Summary
This paper tackles post-training quantization (PTQ) of large language models (LLMs) from an often-overlooked perspective: given a pre-trained high-precision LLM, mainstream sequential quantization frameworks treat all layers uniformly, but this may not be optimal under challenging bit-width settings.
The authors empirically investigate the quantization impact of different layers and make two key observations:
1. Shallow and deep layers are generally more sensitive to quantization than middle layers.
2. Within the shallow/deep layers, the most sensitive ones are the first and last layers, whose quantization errors are significantly larger than those of other layers.
Based on these findings, the authors propose SliderQuant, a novel PTQ framework that adopts a simple adaptive sliding quantization concept.
Extensive experiments on various LLMs demonstrate that the proposed method outperforms existing PTQ approaches in both weight-only quantization and weight-activation quantization.
*Auto-collected on 2026-03-29*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177169404