[论文] LimiX-2: A Contextual Mechanism Network Towards General Structured-Dat...

论文概要

研究领域: ML 作者: Xingxuan Zhang, Gang Ren, Hao Yuan, Hao Zou, Hongze Tan, Hui Wang, Jianhao Song, Jiansheng Li, Jiayao Zhang, Jinghan Zhang, Kaifang Li, Lang Mo, Li Mao, Mingchao Hao, Nuo Xu, Rui Ding, Ruiji Zhang, Shuyang Li, Siyu Mei, Tianyang Zhang, Weiyang Mu, Yancheng Dong, Yongxian Wei, Yuan Xue, Yuanrui Wang, Yue He, Zijia Yang, Ziyun Li, Dongzhe Li, Fuqiang Wang, Jiandong Liu, Jiawei Chen, Jiaxin Du, Kaijie Cheng, Kehan Li, Lei Sun, Linjun Zhou, Ningbo Dai, Qi Wang, Renzhe Xu, Shaoxing Du, Shumeng Yang, Wang Lu, Wenjing Chu, Xiannan Huang, Xiaoyu Lin, Xing Ai, Xinyan Han, Xuanyue Li, Xuanyue Su, Xukun Zhang, Yan Lu, Yaxin Zhang, Yi Qin, Yifei Huang, Yihan Xu, Yongle Lv, Yuanyuan Jiang, Yushan Han, Peng Cui 发布时间: 2026-09-15 arXiv: 2609.17488

中文摘要

我们推出 LimiX-2——LimiX 家族的新模型，通过由我们先前建立的缩放定律指导的模型和数据缩放开发而成。LimiX-2 采用情境机制网络（CMNs）范式，使用情境条件掩蔽建模（CCMM）进行预训练。CMNs 将上下文学习的组织原则从以目标为中心的预测转变为面向机制的联合建模。网络不再围绕传统表格PFN的 p(y|x, D_context) 目标，而是围绕学习 p(x, y| D_context) 设计——这是对数据生成底层结构的上下文依赖表示。预训练使用由结构因果模型（SCM）生成的合成数据集，涵盖多样的图结构、功能机制和观测过程。在 TabArena、TALENT 和 BCCO 上的评估表明，LimiX-2 优于当前特定数据集的模型和表格基础模型。除预测性能外，CMN 范式还促进了 LimiX-2 的因果感知能力：其特征注意力编码了直接的因果关系，能够实现准确的因果骨架恢复。

原文摘要

We introduce LimiX-2, a new model in the LimiX family, developed through model and data scaling guided by our previously established scaling laws. LimiX-2 adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling (CCMM). CMNs shifts the organizing principle of in-context learning from target-centric prediction to mechanism-oriented joint modeling. Rather than centering the network on the \(p(y \mid x, D_{\mathrm{context}})\) objective of conventional tabular PFNs, it is designed around learning \(p(x, y \mid D_{\mathrm{context}})\), a context-dependent representation of the joint structure underlying data generation. Pretraining uses synthetic datasets generated by structural causal models (SCMs) spanning diverse graph structures, funct...

#论文 #arXiv #ML #小凯