[论文] Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Ma...
论文概要
研究领域: NLP
作者: Giannis Daras, Marios Papachristou, Adam Klivans, Antonio Torralba, Constantinos Daskalakis
发布时间: 2026-11-17
arXiv: 2611.23229
中文摘要
为自回归语言模型收集大规模、多样化、高质量的后训练数据集非常困难。因此,监督微调(SFT)通常应用于远小于预训练数据集的规模,导致过拟合和数据分布尾部的灾难性遗忘。为防止这种崩塌,我们提出 Ambient SFT——一个利用不完美预测来扩充微调集的框架。该方法中,预训练模型的"冻结"版本充当噪声教师,微调模型必须在粗粒度层面与其匹配。具体而言,我们要求后训练模型产生的高温度条件分布与预训练的高温度条件分布相匹配。温度超参数控制对预训练模型的信任程度,T→∞ 时退化为常规 SFT,T=1 时为知识蒸馏。实验表明,该方法在领域自适应、指令微调和推理任务上均优于标准 SFT,平均相对提升高达 21%。
原文摘要
Collecting large-scale, diverse, and high-quality post-training datasets for autoregressive language models is challenging. Consequently, Supervised Fine-Tuning (SFT) is typically applied to datasets significantly smaller than those used for pre-training, which leads to overfitting and the catastrophic forgetting of the tails of the data distribution. To prevent this collapse, we introduce Ambient SFT, a principled framework for leveraging imperfect predictions to augment the fine-tuning set. In our method, a "frozen" version of the pre-trained model serves as a noisy teacher that the fine-tuned model must match at a coarse level. In particular, we require that the high-temperature versions of the conditional distributions produced by the post-training model should match the high-temperatu...
自动采集于 2026-11-19
#论文 #arXiv #NLP #小凯