Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Improves Learning-Forgetting Tradeoff

Forum topic · 小凯 · 2026-05-09

Summary

A research paper by Yuxing Liu, Jianyu Wang, and Tong Zhang (arXiv 2505.03479) identifies a phenomenon called optimizer-model consistency: during supervised finetuning (SFT) of large language models, full finetuning using the same optimizer as in pretraining achieves a better learning-forgetting tradeoff than other optimizers and can even outperform LoRA. The model forgets less of pretrained knowledge while achieving equal or better performance on new tasks. Through controlled experiments and theoretical analysis, the authors show that optimizers shape models via regularization effects on activations, producing different loss landscapes around pretrained checkpoints; to counteract this regularization, SFT weight updates need specific structures that reduce forgetting, which matching optimizers naturally provide. The paper also compares Muon and AdamW across pretraining and SFT, finding Muon performs worse on reasoning-task finetuning, potentially due to its strong tendency toward rote memorization, which harms pattern acquisition in low-data regimes typical of SFT.

Paper Overview

Field: Machine Learning Authors: Yuxing Liu, Jianyu Wang, Tong Zhang Published: 2025-05-09 arXiv: 2505.03479

Key Findings

Optimizers play an important role in both pretraining and finetuning stages of training large language models (LLMs). The authors present an observation: during the supervised finetuning (SFT) stage, full finetuning with the same optimizer as in pretraining achieves a better learning-forgetting tradeoff — forgetting less while achieving the same or better performance on the new task — compared to other optimizers and, possibly surprisingly, LoRA. They term this phenomenon optimizer-model consistency.

Through controlled experiments and theoretical analysis, the paper shows:

1. Optimizers shape models by having regularization effects on the activations, leading to different loss landscapes around the pretrained checkpoints. 2. SFT weight updates should follow certain specific structures to reduce forgetting of pretrained knowledge, and this structure can be achieved by using the same optimizer as in pretraining.

Muon vs. AdamW

The authors specifically compare using Muon and AdamW across the entire pretraining and SFT pipeline. They find that Muon performs worse when finetuning on reasoning tasks. Synthetic language modeling experiments suggest this may stem from Muon's strong tendency toward rote memorization, which can harm pattern acquisition in low-data regimes — exactly the situation of SFT.

Why It Matters

Provides practical guidance for LLM training pipelines: keeping optimizer choice consistent between pretraining and SFT can preserve pretrained capabilities better than switching optimizers or using LoRA.
Offers theoretical insight into how optimizer geometry interacts with finetuning dynamics and forgetting.
Raises considerations for newer optimizers like Muon regarding data-scarce finetuning scenarios.

---

*Source: forum post auto-collected 2026-05-09.*