[论文] PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization

论文概要

研究领域: cs.LG
作者: Yuchen Yang, Yifan Zhao, Anisha Dasgupta
发布时间: 2026-07-21
arXiv: 2507.15488

中文摘要

混合专家（MoE）是一类流行的大语言模型，兼具高效率和高精度。然而，在KV缓存密集的服务场景中，MoE常常在模型权重的GPU内存需求与不断增长的KV缓存之间存在张力。我们提出PagedWeight，一种用于MoE LLM服务的新型管理方法，在运行时动态量化MoE模型的权重，并平衡专家权重精度与KV缓存大小。PagedWeight揭示并有效导航模型任务精度、内存消耗和吞吐量/延迟之间的复杂权衡。在多个内存敏感的MoE服务场景中，PagedWeight相比现有量化基线改善了精度-内存权衡。PagedWeight在实现与FP16等效精度的同时，节省高达72.0%的GPU内存并提升1.94倍吞吐量，在相似内存预算下比量化方法提升最多39.3%的质量，且吞吐量损失最多仅4.1%。

原文摘要

Mixture-of-Experts (MoE) is a popular class of large language models (LLMs), offering high efficiency and accuracy. However, in KV-cache-intensive serving scenarios, MoEs often exhibit a tension between the GPU memory requirements of the model weights and the growing KV cache. We propose PagedWeight, a novel management method for MoE LLM serving that dynamically quantizes MoE model's weights at runtime and balances expert-weight precision with the KV cache sizes. PagedWeight exposes and effectively navigates the complex tradeoff between the model's task accuracy, memory consumption, and throughput/latency. Across several memory-sensitive MoE serving scenarios, PagedWeight improves the quality-memory tradeoff over several existing quantization baselines. PagedWeight achieves FP16-equivalent accuracy with up to 72.0% GPU memory savings and 1.94x throughput improvement, and improves quality over quantization methods by up to 39.3% at a similar memory budget with at most 4.1% throughput loss.

自动采集于 2026-07-21

#论文 #arXiv #LG #小凯