[论文] Pretraining Data Scale Reverses the Muon-AdamW Fine-Tuning Gap

论文概要

研究领域: ML 作者: Catherine Li, Jingchu Gai, Andrej Risteski, Jacob Mitchell Springer, Aditi Raghunathan 发布时间: 2026-11-02 arXiv: 2611.03633

中文摘要

Muon 近期作为 LLM 预训练的新兴优化器受到关注，相比 AdamW 在降低预训练损失上有加速效果。但 Muon 的优势在微调后是否仍然存在尚不清楚。本文研究 Muon 与 AdamW 的下游行为，揭示了一个出人意料的趋势：当预训练数据预算超过一定规模后，AdamW 在微调后反而优于 Muon。并且模型越大，AdamW 反超所需的 token-参数比越小。为解释这一反转，我们将微调视作对输出的微扰，引入 margin-to-spread ratio（正确 logits 高出错误 logits 的程度相对于错误 logits 散布宽度的比值）。在语言模型中，小数据预算下 Muon 的该比值更大，大数据预算下 AdamW 更大——与微调趋势一致。在线性联想记忆模型中，我们证明一次更新后梯度下降的该比值随数据预算增长会超过 Muon；仿真显示这一趋势在长时间训练后仍保持。

原文摘要

Muon has recently emerged as a promising optimizer for pretraining large language models, offering a speedup over AdamW in minimizing pretraining loss. However, it remains unclear whether Muon's advantage persists after fine-tuning. In this paper, we study the downstream behavior of Muon and AdamW and uncover a surprising trend: beyond a sufficiently large pretraining data budget, AdamW outperforms Muon after fine-tuning. Moreover, as model size increases, AdamW overtakes Muon at progressively smaller pretraining token-parameter ratios. To explain this reversal, we view fine-tuning as a perturbation of the outputs and introduce the margin-to-spread ratio, which measures how far the correct logit lies above the wrong ones relative to how widely the wrong logits are spread. In language model...

#论文 #arXiv #ML #小凯