English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original
智柴网 中文原文

Optimizer-Model Consistency: Full Finetuning with the Pretraining Optimizer Improves Learning-Forgetting Tradeoffs

Forum topic · 小凯 · 2026-05-10

Summary

This paper introduces 'optimizer-model consistency': full supervised finetuning (SFT) of large language models using the same optimizer as pretraining achieves a better learning-forgetting tradeoff than switching optimizers, retaining more pretrained knowledge while matching or exceeding performance on new tasks—and sometimes outperforming LoRA. The authors Yuxing Liu, Jianyu Wang, and Tong Zhang support this observation with controlled experiments and theoretical analysis. They show that optimizers shape models through a regularization effect on activations, producing different loss landscapes around the pretrained checkpoint, and that SFT weight updates should follow specific structures to reduce forgetting of pretrained knowledge—a structure naturally achieved by reusing the same optimizer. The paper specifically compares Muon and AdamW across pretraining and SFT stages, finding that Muon performs worse when finetuning on reasoning tasks. A synthetic language modeling experiment suggests this stems from Muon's strong tendency toward memorization, which can impair pattern acquisition in small-data regimes such as SFT. Source: arXiv 2605.06654, posted 2026-05-07 on zhichai.net.

Paper Overview

Field: Machine Learning Authors: Yuxing Liu, Jianyu Wang, Tong Zhang Published: 2026-05-07 arXiv: 2605.06654

Abstract (Translated)

Optimizers play an important role in both the pretraining and finetuning stages of large language model (LLM) training. In this paper, we make an observation: during the supervised finetuning (SFT) stage, full finetuning with the same optimizer used in pretraining achieves a better learning-forgetting tradeoff—forgetting less while attaining equal or better performance on new tasks—compared to other optimizers, and sometimes surprisingly outperforms LoRA. We call this phenomenon optimizer-model consistency.

To better understand it, through controlled experiments and theoretical analysis, we demonstrate:

1. Optimizers can shape models through a regularization effect on activations, leading to different landscapes around the pretraining checkpoint. 2. In response to this regularization effect, weight updates during SFT should follow certain specific structures to reduce forgetting of knowledge learned in pretraining, which can be achieved by using the same optimizer.

Additionally, we specifically compare Muon and AdamW across both the pretraining and SFT stages, finding that Muon performs worse when finetuning on reasoning tasks. Through a synthetic language modeling experiment, we show that this may stem from Muon's strong tendency toward rote memorization, which can harm pattern acquisition in small-data regimes—as is the case in SFT.

---

*Auto-collected on 2026-05-10*

Tags

#optimizer#llm-finetuning#supervised-finetuning#lora#muon#adamw#catastrophic-forgetting#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177619694