论文概要
研究领域: ML 作者: Yao Tong, Jiayuan Ye, Anastasia Borovykh 发布时间: 2025-04-17 arXiv: 2504.13085
中文摘要
语言模型是否能够系统性泛化仍然是一个活跃争论的话题。然而，经验性能受多种因素共同影响，如训练数据、训练范式和推理时策略，这使得失败难以解释。我们引入一个基于最短路径规划的受控合成环境——这是一个经典的可组合序列优化问题。该设置能够清晰分离这些因素，并支持两个正交的泛化维度：空间迁移到未见过的地图，以及长度扩展到更长跨度的问题。我们发现模型展现出强大的空间迁移能力，但由于递归不稳定性而在长度扩展上持续失败。我们进一步分析了学习流程中不同阶段如何影响系统性问题解决：例如，数据覆盖设定了能力上限；强化学习提高了训练稳定性但并未扩展这些上限；推理时扩展增强了性能但无法挽救长度扩展失败。
原文摘要
Whether language models can systematically generalize remains actively debated. Yet empirical performance is jointly shaped by multiple factors such as training data, training paradigms, and inference-time strategies, making failures difficult to interpret. We introduce a controlled synthetic environment based on shortest-path planning, a canonical composable sequential optimization problem. The setup enables clean separation of these factors and supports two orthogonal axes of generalization: spatial transfer to unseen maps and length scaling to longer-horizon problems. We find that models exhibit strong spatial transfer but consistently fail under length scaling due to recursive instability. We further analyze how distinct stages of the learning pipeline influence systematic problem-solv...
--- *自动采集于 2026-04-18*
#论文 #arXiv #ML #小凯