Summary
A new paper from Tri Dao's team (authors of FlashAttention and Mamba), titled ScaleSearch, shows that the standard practice of choosing a quantization scale factor as the maximum absolute value within a block is suboptimal for low-precision GPU compute such as NVFP4. ScaleSearch instead searches for the optimal scale factor, exploiting mantissa bits in microscaling formats to minimize quantization error, and integrates seamlessly into existing PTQ pipelines and low-precision attention methods. Reported results include a 27% reduction in quantization error for NVFP4, a 15-point PTQ improvement for Qwen3-8B on MATH500, up to 0.77 perplexity improvement for Llama 3.1 70B on Wikitext-2, and ScaleSearchAttention achieving near-lossless NVFP4 attention. The core insight: hardware defaults are often not optimal, and simply searching instead of taking defaults yields meaningful gains. Reference: arXiv:2605.12464, Tri Dao, Daniel Y. Fu et al.
A new paper from Tri Dao's team (the authors of FlashAttention and Mamba). The problem is simple: when GPUs perform low-precision computation (e.g., NVFP4), each data block must be assigned a scale factor. The standard approach takes the maximum absolute value within the block as the scale. The paper proves this default choice is suboptimal.
The core innovation, ScaleSearch, searches for the optimal scale factor instead of using the default maximum. It leverages the mantissa bits in microscaling formats to minimize quantization error, and can be seamlessly integrated into existing PTQ and low-precision attention methods.
Key results:
- Quantization error reduced by 27% (NVFP4)
- 15-point PTQ improvement for Qwen3-8B on MATH500
- Wikitext-2 PPL for Llama 3.1 70B improved by up to 0.77
- ScaleSearchAttention proposed, achieving near-zero loss for NVFP4 attention
*Core insight*: hardware default parameters are often not optimal — as long as you're willing to
search rather than
take the default, you can squeeze out substantial performance gains.
Reference: ScaleSearch / arXiv:2605.12464, Tri Dao, Daniel Y. Fu et al.
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177619956