论文概要
研究领域: ML 作者: Yuxing Liu, Jianyu Wang, Tong Zhang 发布时间: 2025-05-09 arXiv: 2505.03479
Summary
Optimizers play an important role in both pretraining and finetuning stages when training large language models (LLMs). In this paper, the authors present an observation that full finetuning with the same optimizer as in pretraining achieves a better learning-forgetting tradeoff, i.e., forgetting less while achieving the same or better performance on the new task, than other optimizers and, possibly surprisingly, LoRA, during the supervised finetuning (SFT) stage. They term this phenomenon optimizer-model consistency.
To better understand it, through controlled experiments and theoretical analysis, the paper shows that:
1. Optimizers can shape the models by having regularization effects on the activations, leading to different landscapes around the pretrained checkpoints. 2. In response to this regularization effect, weight updates in SFT should follow certain specific structures to reduce forgetting of pretrained knowledge, and this can be achieved by using the same optimizer.
Muon vs. AdamW
The authors specifically compare using Muon and AdamW throughout pretraining and full-stage SFT, finding that Muon performs worse when finetuning on reasoning tasks. Through synthetic language modeling experiments, they demonstrate this may stem from Muon's strong tendency toward rote memorization, which can hurt pattern acquisition under limited data—as is the case in SFT.
Key points
- Same-optimizer full finetuning (optimizer-model consistency) reduces forgetting during SFT while matching or beating other optimizers and LoRA on new tasks.
- Optimizers impose regularization effects on activations, shaping the loss landscape around pretrained checkpoints.
- Structured SFT weight updates are key to mitigating catastrophic forgetting.
- Muon shows a memorization bias that hurts reasoning-task finetuning with limited data compared to AdamW.
*Links*: arXiv:2505.03479 *Auto-collected from zhichai.net on 2026-05-09.*