[论文] Thermodynamic Response Functions in Singular Bayesian Models

论文概要

研究领域: ML 作者: Anonymous 发布时间: 2026-03-06 arXiv: 2603.05500

中文摘要

奇异统计模型（包括混合模型、矩阵分解和神经网络）由于参数不可识别性和退化的费雪几何而违反正则渐近理论。本文表明，后验温度调节诱导了后验分布的单参数变形，其相关可观测量产生了热力学响应函数的层级结构。一个通用的协方差恒等式将温度调节期望的导数与后验波动联系起来，将 WAIC、WBIC 和奇异波动置于统一的响应框架内。研究结果表明，热力学响应理论为解释奇异贝叶斯学习中的复杂性、预测变异性和结构重组提供了一个自然的组织框架。

原文摘要

Singular statistical models-including mixtures, matrix factorization, and neural networks-violate regular asymptotics due to parameter non-identifiability and degenerate Fisher geometry. We show that posterior tempering induces a one-parameter deformation of the posterior distribution whose associated observables generate a hierarchy of thermodynamic response functions. A universal covariance identity links derivatives of tempered expectations to posterior fluctuations, placing WAIC, WBIC, and singular fluctuation within a unified response framework. Our results suggest that thermodynamic response theory provides a natural organizing framework for interpreting complexity, predictive variability, and structural reorganization in singular Bayesian learning.

#论文 #arXiv #ML #小凯