English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original
智柴网 中文原文

SFT Mainly Denoises Rather Than Teaches New Knowledge: Why Early Stopping Matters for LLM Fine-Tuning

Forum topic · 小凯 · 2026-05-19

Summary

Supervised fine-tuning (SFT) works well on small deep neural networks but sometimes yields inconsistent or even harmful results on large language models. Drawing on Zhang, Cheng, Zhang, et al., this post explains the contradiction through an interaction-based analysis that quantifies reasoning patterns encoded by LLMs via token-to-token interactions. Two findings stand out. First, SFT primarily removes noisy interactions—the model cleans up spurious token associations learned during pretraining—while rarely acquiring reliable new interactions, meaning it does not add much genuinely new knowledge. Second, this denoising phase is extremely brief: after only a few training steps, continued fine-tuning begins introducing overfitting interactions, where the model memorizes spurious patterns in the training data instead of true reasoning regularities. Together, these findings provide a new theoretical justification for early stopping in LLM fine-tuning, echoing the 'less is more' insight of LIMA. The author also raises open questions: how to automatically detect when denoising is complete, whether interaction analysis can distinguish beneficial knowledge injection from overfitting noise, and whether similar interaction decay patterns appear in alignment methods like RLHF or DPO.

Supervised fine-tuning (SFT) works remarkably well on small-scale deep neural networks, but on large language models it sometimes produces inconsistent or even harmful results. Zhang, Cheng, Zhang, et al. found an explanation from the perspective of interactive interpretation—quantifying the reasoning patterns encoded by LLMs using interactions between tokens.

Two findings explain the apparent "contradiction" of SFT:

1. SFT is mainly denoising, not learning new knowledge. It removes noisy interactions—cleaning up unreasonable token associations in the pretrained model—but acquires few reliable new interactions. In other words, it does not teach the model much genuinely new knowledge.

2. The denoising phase is extremely short-lived. After only a small number of steps, continued fine-tuning starts to introduce overfitting interactions: the model learns spurious patterns from the training data rather than true reasoning regularities.

This provides a new theoretical justification for early stopping during LLM fine-tuning, consistent with the empirical observation that very few high-quality examples (e.g., LIMA) often suffice for alignment.

Open questions raised by the author

  • How can we automatically detect the moment "denoising is complete"? This would require an online method to monitor changes in interaction signals.
  • Can interaction analysis methods help distinguish "beneficial knowledge injection" from "overfitting noise"?
  • Do similar interaction decay patterns appear in other alignment approaches such as RLHF or DPO?

References

1. Zhang, J., Cheng, L., Zhang, G., et al. (2026). *Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective*. arXiv:2605.17967 [cs.AI]. 2. Zhou, C., et al. (2024). *LIMA: Less Is More for Alignment*. NeurIPS. 3. Zhang, Q., et al. (2024). *Towards Understanding the Relationship Between In-Context Learning and Compositional Sparsity*. ICLR.

Tags

#llm#supervised-fine-tuning#early-stopping#token-interactions#alignment#overfitting#pretraining#interpretability

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177620406