[论文] Null Controls Are Necessary but Not Sufficient: Evaluating Acceptan...

论文概要

研究领域: ML 作者: Tianyu Wu 发布时间: 2026-10-19 arXiv: 2610.22445

中文摘要

自动发现系统提出科学解释并检验哪些可接受——但通过检验是否就确立了主张？本文开发四个可执行操作改变物理描述：识别冗余（L1）、动力学粗粒化（L2）、结构迁移（L3）、由守恒不平衡推断缺失量（L4），并构造三种评估准则：零接受（NAR）、正接受（PAR）、后果一致（CAR）。一个把所有状态映射为零的「归约」能完美预测自身未来，在 129 个公开混沌系统的 96% 上通过封闭性检验，尽管未保留任何动力学变异；独立的可观测保持检验则全程拒绝它。在受控基准上，组合准则接受 24/24 真实归约、拒绝 72/72 零假设。其余操作能区分对称与冗余、验证迁移、拒绝虚假的守恒不平衡。在语言模型与 ESM-2 蛋白模型中，所提对应关系被随机旋转后线性 CKA 纹丝不动，而局部干预能区分真伪映射；在稀疏自编码器中，富集检验拒绝全部 16 个零假设，但独立消融只支持 45% 可评估的富集特征（匹配对照仅 0.5%）。结果揭示两个「接受缺口」：描述性检验可能接受科学上空洞的候选；被接受的候选可能缺乏独立验证的后果。可靠的自动发现必须同时评估「检验接受了什么」与「确立了什么科学后果」。

原文摘要

Automated discovery systems propose scientific explanations and test which to accept. Does passing a test establish the intended claim? We develop four executable operations for changing physical descriptions: identifying redundancies (L1), coarse-graining dynamics (L2), transferring structure (L3), and inferring missing quantities from conservation imbalances (L4). We evaluate these operations and three interpretability settings through null acceptance (NAR), positive acceptance (PAR), and consequence agreement (CAR). A reduction mapping every state to zero predicts its own future perfectly. It passes closure on 96% of 129 public chaotic systems despite retaining no dynamical variation. An independent observable-preservation test rejects it throughout. On controlled benchmarks, the combined criteria accept 24/24 genuine reductions and reject 72/72 nulls. The other operations distinguish symmetries from redundancies, validate transfers, and reject spurious conservation imbalances. In language and ESM-2 protein models, linear CKA remains unchanged when a proposed correspondence is randomly rotated, whereas localized interventions distinguish genuine mappings from tested nulls. In sparse autoencoders, enrichment rejects all 16 tested nulls, yet independent ablation supports 45% of evaluable enriched features, compared with 0.5% of matched controls. These results reveal two acceptance gaps: descriptive tests can accept scientifically empty candidates, and accepted candidates can lack independently verified consequences. Reliable automated discovery requires evaluating both what a test accepts and the scientific consequences it establishes. Code and data: this https URL.

---

*自动采集于 2026-10-19*

#论文 #arXiv #ML #小凯