Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less

Forum topic · 小凯 · 2026-05-09

Summary

This paper (arXiv:2505.03479) by Yuxing Liu, Jianyu Wang, and Tong Zhang introduces the concept of optimizer-model consistency in large language model training. The authors observe that during supervised finetuning (SFT), full finetuning with the same optimizer used during pretraining achieves a better learning-forgetting tradeoff: the model forgets less of its pretrained knowledge while matching or exceeding the performance of alternative optimizers, and in some cases even outperforming LoRA. Through controlled experiments and theoretical analysis, they show that optimizers shape models by imposing regularization effects on activations, producing distinct loss landscapes around pretrained checkpoints. SFT weight updates therefore need certain structures to reduce forgetting, which using the same optimizer naturally provides. The authors specifically compare Muon and AdamW across pretraining and SFT, finding that Muon performs worse on reasoning-task finetuning. Synthetic language modeling experiments suggest Muon's strong tendency toward rote memorization may impair pattern acquisition when data is scarce, as in typical SFT settings. The findings offer practical guidance for choosing optimizers across the pretraining-finetuning pipeline.

Paper Overview

Research Area: Machine Learning Authors: Yuxing Liu, Jianyu Wang, Tong Zhang Published: 2025-05-09 arXiv: 2505.03479

Abstract

Optimizers play an important role in both pretraining and finetuning stages when training large language models (LLMs). In this paper, the authors present an observation that full finetuning with the same optimizer as in pretraining achieves a better learning-forgetting tradeoff, i.e., forgetting less while achieving the same or better performance on the new task, than other optimizers and, possibly surprisingly, LoRA, during the supervised finetuning (SFT) stage. This phenomenon is termed optimizer-model consistency.

To better understand it, through controlled experiments and theoretical analysis, the paper shows that:

1. Optimizers can shape the models by having regularization effects on the activations, leading to different landscapes around the pretrained checkpoints. 2. In response to this regularization effect, weight updates in SFT should follow certain specific structures to reduce forgetting of pretrained knowledge, which can be achieved by using the same optimizer.

Additionally, the authors specifically compare Muon and AdamW used throughout both pretraining and SFT, finding that Muon performs worse when finetuning on reasoning tasks. Through synthetic language modeling experiments, they demonstrate that this may stem from Muon's strong tendency toward rote memorization, which can harm pattern acquisition under limited data — as is the case in SFT.

Key Findings

Optimizer-model consistency: Reusing the pretraining optimizer for full finetuning yields less forgetting and equal or better new-task performance than alternative optimizers, and can even outperform LoRA.
Mechanism: Optimizers impose regularization effects on activations, shaping distinct loss landscapes around pretrained checkpoints; matching optimizers produces weight updates with structures that preserve prior knowledge.
Muon vs. AdamW: Muon underperforms on reasoning-task finetuning, likely due to a rote-memorization bias that hurts pattern acquisition in low-data regimes.

--- *Auto-collected on 2026-05-09*