[论文] Marformer: A Transformer for Predicting Missing Data Distributions

论文概要

研究领域: ML 作者: Prabhav Singh, Xiheng Tom Wang, Haojun Shi, Jason Eisner 发布时间: 2026-10-08 arXiv: 2610.12379

中文摘要

真实决策总是在不完整信息下做出：只能观测到部分随机变量时，需要预测其余变量。缺失变量的条件边际分布是计算贝叶斯风险和情报价值（VOI）——即决策前多获取一次观测的期望收益——的关键成分。我们提出 Marformer：一个训练来直接预测给定任意观测集合时条件边际的 Transformer。与 BERT 从上下文预测缺失词类似，Marformer 为每个分布 p(X_i) 构建隐向量表示，并通过对其余分布 p(X_j) 的注意力迭代细化。与生成式方法不同，Marformer 不建模完整联合分布、不需要数据生成过程的领域知识、单次前向传播即可完成全部预测。在贝叶斯网络、离散化多元高斯、结构化标注数据三个含缺失数据的合成领域上，Marformer 即使面对已知真实模型族和先验的经典缺失数据方法也能匹敌或超越；在真实标注数据集上，最大训练规模下优于所有基线。两种情形下，Marformer 都比生成式基线快得多。

原文摘要

Real decisions are made under incomplete information. If we observe only some of the random variables we need, we can predict the others. The \textbf{conditional marginals} over the missing variables are the key ingredient for computing Bayes risk and Value of Information (VOI), the expected gain from acquiring one more observation before deciding. We present the Marformer, a Transformer trained to directly predict conditional marginals given any set of observed values. Like BERT, which is trained to predict missing words from context, the Marformer constructs a hidden-vector representation for each distribution \(p(X_i)\) and iteratively refines it through attention to other distributions \(p(X_j)\). Unlike generative approaches, the Marformer does not model the full joint distribution, requi...

#论文 #arXiv #ML #小凯