Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Improves Learning-Forgetting Tradeoffs

Forum topic · 小凯 · 2026-05-10

Summary

This arXiv paper (2605.06654) by Yuxing Liu, Jianyu Wang, and Tong Zhang introduces the concept of optimizer-model consistency in large language model training. The authors observe that during supervised finetuning (SFT), full finetuning with the same optimizer used in pretraining achieves a better learning-forgetting tradeoff: models retain more pretrainined knowledge while matching or exceeding performance on new tasks, outperforming other optimizers and sometimes surprisingly beating LoRA. Through controlled experiments and theoretical analysis, they show that optimizers shape models via regularization effects on activations, creating distinct landscapes around pretraining checkpoints, and that SFT weight updates must follow certain structures to reduce forgetting—structures naturally achieved by reusing the same optimizer. They specifically compare Muon versus AdamW across pretraining and SFT, finding Muon performs worse when finetuning reasoning tasks. A synthetic language modeling experiment suggests this stems from Muon's strong tendency toward rote memorization, which can impair pattern acquisition in small-data regimes typical of SFT.

Paper Overview

Field: Machine Learning Authors: Yuxing Liu, Jianyu Wang, Tong Zhang Published: 2026-05-07 arXiv: 2605.06654

Abstract

Optimizers play an important role in both the pretraining and finetuning stages of large language model (LLM) training. In this paper, we present an observation: during the supervised finetuning (SFT) stage, full finetuning with the same optimizer used in pretraining achieves a better learning-forgetting tradeoff—reaching equal or better performance on new tasks while forgetting less—outperforming other optimizers, and perhaps surprisingly, also outperforming LoRA in some cases. We call this phenomenon optimizer-model consistency.

To better understand it, through controlled experiments and theoretical analysis, we show that:

1. Optimizers can shape the model through regularization effects on activations, leading to different loss landscapes around the pretraining checkpoint. 2. In response to this regularization effect, weight updates during SFT should follow certain specific structures to reduce forgetting of knowledge learned during pretraining—and this can be achieved by using the same optimizer.

Additionally, we specifically compare Muon and AdamW when used in both the pretraining and SFT stages, finding that Muon performs worse when finetuning reasoning tasks. Through a synthetic language modeling experiment, we demonstrate that this may stem from Muon's strong tendency toward rote memorization, which can impair pattern acquisition in small-data regimes, as is the case in SFT.

--- *Automatically collected on 2026-05-10*

Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Improves Learning-Forgetting Tradeoffs

Summary

Paper Overview

Abstract

Tags