[论文] Conflicting Supervision Moves Commitment, Not Capability: A 12.29σ arr...

论文概要

研究领域: ML
作者: Wenhui Chen
发布时间: 2026-10-05
arXiv: 2610.00234

中文摘要

用两种互不兼容但都正确的约定写同一批问题来训练模型，然后问数据顺序往参数里写下了什么。学习率时间表不是背景条件——它是取平均的操作符，它决定答案。我们证明一个上界：排列与时间表作为独立相乘因子进入排序效应——排列只以块周期进入，时间表只以「终点能给任一时刻赋多大权重」进入。衰减时间表无法在同一时刻给出大步长加未收缩余项；常数时间表恰在最后一步做到。衰减削弱排序效应在预训练中已有报道；机制、分离及两半受控测量是我们的贡献。同一语料十种排列、同一预算、除路径外全固定，在两个仅 lr_scheduler_type 不同的族下各跑两次：常数率下内部分配差跨 0.2221、11.63 个对比下限、随分块程度单调；单一余弦（所有已发表结果所用）下同样十臂只占两个可区分状态，而其自身分辨率本可区分约十个。「顺序重要」与「顺序不重要」是同一旋钮两端。路径写下的是模型承诺采纳哪种约定，任何精确匹配基准都看不见。十二臂 acc_A+acc_B 恒定至 9.7% 以内而分配份额从 0.04 到 0.87——因此本文测得的 12.29σ 排列切换在约定无关指标下恰好为零。该守恒引用自衰减族，常数率族读数更嘈杂。提示中标注约定使切换坍缩并达并集上限的 87.5%。

原文摘要

"Train a model on the same problems written under two incompatible conventions, both correct, and ask what the ordering of that data writes into the parameters. The learning-rate schedule is not a background condition for that question. It is the averaging operator, and it decides the answer. We prove a bound in which the arrangement and the schedule enter the ordering effect as separate multiplied factors: the arrangement only as a block period, the schedule only as how much weight the endpoint can place on any one moment of the run. A decaying schedule cannot put a large step size and an uncontracted remainder at the same moment; a constant one does exactly that at the last step. That decay moderates ordering effects has been reported in pretraining; the mechanism, the separation, an...

自动采集于 2026-10-05

#论文 #arXiv #ML #小凯