[论文] Predictive Credit: Measuring What Scientific Explanations Add to Exper...

论文概要

研究领域: ML
作者: Jingjie Ning, Xueqi Li, Yibo Kong, Dongting Li
发布时间: 2026-10-05
arXiv: 2610.00314

中文摘要

研究智能体解释计划中的实验。我们以共享干预、预测者和结果的成对预测测量「预测信用」，变化描述、匹配解释和供体上下文。五项检查追踪承诺、交付、预测增益、对齐和已知信号采纳。受控学习 336 个前瞻状态、12 个 Tox21 端点、24 个 OpenML 任务上，v5 冻结信用判定不确定。Tox21 预注册 ROC AUC 区间分数损害检验未达（D-M=-.0026，95% 区间 [-.0174, .0104]）；OpenML 联合形成、点等价和可重复性规则未达。匹配解释相对描述的点准确率增益未确认，种子供体区间跨零。DeepSeek V4 Pro 下匹配卡片和供体卡片将 Tox21 次级漂移分别降 64.5% 和 59.1%。DeepSeek V4 Flash 回放将匹配点 MAE 从 .01823 升至 .02020，未达匹配供体区间等价。OpenML 全卡片使名义 80% 区间加宽 21%，覆盖率 49.3% 对描述的 51.4%（144 卡中 66 卡）。直接文本 Flash 交付全部 144 条笔记但无匹配点准确率增益。研究者撰写的机制阳性对照相对描述降点 MAE 2.60 个百分点。该协议为研究智能体基准和科学预测测量预测信用；自然解释信用在所测供体分辨率下未确认。

原文摘要

Research agents explain planned experiments. We measure predictive credit with paired forecasts sharing an intervention, forecaster, and outcome while varying description, matched explanation, and donor context. Five checks track commitment, delivery, predictive gain, alignment, and known-signal uptake. Across 336 prospective states in controlled learning, 12 Tox21 endpoints, and 24 OpenML tasks, v5's frozen credit decision was inconclusive. Tox21's preregistered ROC AUC interval-score harm test was unmet (\(D-M=-.0026\), 95 percent interval [\(-.0174\), .0104]); OpenML's joint formation, point-equivalence, and repeatability rule was unmet. Matched point-accuracy gains over description remained unconfirmed, and Tox21/OpenML seed-donor intervals spanned zero. Under requested DeepSee...

自动采集于 2026-10-05

#论文 #arXiv #ML #小凯