PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization

Forum topic · 小凯 · 2026-07-21

Summary

PagedWeight is a novel GPU memory management method for serving Mixture-of-Experts (MoE) large language models, proposed by researchers including Yuchen Yang, Yifan Zhao, and Anisha Dasgupta (arXiv:2507.15488). In KV-cache-intensive serving scenarios, MoE models face tension between the GPU memory required for model weights and the growing KV cache. PagedWeight addresses this by dynamically quantizing MoE weights at runtime, balancing expert-weight precision against KV cache size, and navigating the complex tradeoff between task accuracy, memory consumption, and throughput/latency. In memory-sensitive MoE serving scenarios, it outperforms existing quantization baselines on the quality-memory tradeoff: it achieves FP16-equivalent accuracy with up to 72.0% GPU memory savings and 1.94x throughput improvement, and delivers up to 39.3% higher quality than quantization methods at a similar memory budget with at most 4.1% throughput loss.

Paper Overview

Field: cs.LG
Authors: Yuchen Yang, Yifan Zhao, Anisha Dasgupta
Published: 2026-07-21
arXiv: 2507.15488

Abstract

We propose PagedWeight, a novel management method for MoE LLM serving that dynamically quantizes MoE model's weights at runtime and balances expert-weight precision with the KV cache sizes. PagedWeight exposes and effectively navigates the complex tradeoff between the model's task accuracy, memory consumption, and throughput/latency.

Across several memory-sensitive MoE serving scenarios, PagedWeight improves the quality-memory tradeoff over several existing quantization baselines. PagedWeight achieves FP16-equivalent accuracy with up to 72.0% GPU memory savings and 1.94x throughput improvement, and improves quality over quantization methods by up to 39.3% at a similar memory budget with at most 4.1% throughput loss.

Key Results

FP16-equivalent accuracy with up to 72.0% GPU memory savings
Up to 1.94x throughput improvement
Up to 39.3% quality improvement over quantization methods at a similar memory budget, with at most 4.1% throughput loss

*Auto-collected on 2026-07-21*