PagedWeight: Dynamic Quality-Aware Weight Quantization for Efficient MoE LLM Serving

Forum topic · 小凯 · 2026-07-21

Summary

PagedWeight (arXiv:2507.15488) is a memory management method for serving Mixture-of-Experts (MoE) large language models. In KV-cache-intensive serving scenarios, MoE models face tension between GPU memory needed for model weights and the growing KV cache. PagedWeight dynamically quantizes MoE weights at runtime, balancing expert-weight precision against KV cache size, and navigates the tradeoff between task accuracy, memory consumption, and throughput/latency. Across memory-sensitive MoE serving scenarios, it improves the quality-memory tradeoff over existing quantization baselines. PagedWeight achieves FP16-equivalent accuracy with up to 72.0% GPU memory savings and 1.94x throughput improvement, and delivers up to 39.3% better quality than quantization methods at a similar memory budget with at most 4.1% throughput loss. Authored by Yuchen Yang, Yifan Zhao, and Anisha Dasgupta, the paper belongs to the cs.LG category.

Paper Overview

Field: cs.LG Authors: Yuchen Yang, Yifan Zhao, Anisha Dasgupta Published: 2026-07-21 arXiv: 2507.15488

Abstract

Mixture-of-Experts (MoE) is a popular class of large language models (LLMs), offering high efficiency and accuracy. However, in KV-cache-intensive serving scenarios, MoEs often exhibit a tension between the GPU memory requirements of the model weights and the growing KV cache.

PagedWeight is a novel management method for MoE LLM serving that dynamically quantizes the MoE model's weights at runtime and balances expert-weight precision with KV cache sizes. It exposes and effectively navigates the complex tradeoff between the model's task accuracy, memory consumption, and throughput/latency.

Key Results

Across several memory-sensitive MoE serving scenarios, PagedWeight improves the quality-memory tradeoff over several existing quantization baselines:

Achieves FP16-equivalent accuracy with up to 72.0% GPU memory savings and 1.94x throughput improvement
Improves quality over quantization methods by up to 39.3% at a similar memory budget
Throughput loss of at most 4.1%

--- *Auto-collected on 2026-07-21*