[论文] Mathematical Transfer in LLMs Follows Reasoning Approach More Than Top...

论文概要

研究领域: ML 作者: Sajad Goudarzi, Samaneh Zamanifard, Seyed Amin Seyed Haeri, Moloud Nasiri, Hamed Rahimian 发布时间: 2026-10-05 arXiv: 2610.00331

中文摘要

为 LLM 选数学训练数据时，自然组织原则是主题：给概率目标选概率例题。另一选择是推理方法：共享解题方法的成题解答，即使领域不同。我们问哪种关系微调后产生更大迁移。两个平衡交叉 2×2 设计：概率×组合交叉不变量推理×双重计数（2,000 题），数论×几何交叉补集×鸽巢推理（800 题）。每设计中每个单元轮流作留出目标：同方法（SA）源共享方法但换主题，同主题（ST）源共享主题但换方法。每源在每个角色各出现一次，加性源质量效应从等权聚合对比中抵消。五个基础模型×每设计三个种子，SA 在全部 40 个种子池化比较中优于 ST。主设计模型级优势 8.2–16.2 个百分点（均值 10.8），第二设计 12.0–16.0（均值 14.3）；十个模型级 95% 置信区间全排除零。两个设计中 ST 源在嵌入和词汇度量下与目标更相似——SA 优势与测得的陈述级相似度排序相反。这些发现确认：所评估组合中推理方法是比主题更有效的数学迁移匹配标准。

原文摘要

When selecting mathematical training data for LLMs, a natural organizing principle is topic: probability examples for probability targets. An alternative is reasoning approach: worked solutions that share a solution method with the target, even when the mathematical domain differs. We ask which relation produces greater transfer after fine-tuning. We evaluate two counterbalanced \(2\times2\) designs: probability and combinatorics crossed with invariant reasoning and double counting (2,000 problems), and number theory and geometry crossed with complement and pigeonhole reasoning (800 problems). In each design, every cell serves as the held-out target in turn: same-approach (SA) sources share the target's method but change the topic, while same-topic (ST) sources share the topic but change...

#论文 #arXiv #ML #小凯