Thermodynamic Response Functions in Singular Bayesian Models

Forum topic · 小凯 · 2026-03-07

Summary

This arXiv paper (2603.05500) studies singular statistical models—including mixture models, matrix factorization, and neural networks—which violate regular asymptotic theory due to parameter non-identifiability and degenerate Fisher geometry. The authors show that posterior tempering induces a one-parameter deformation of the posterior distribution, and the observables associated with this deformation generate a hierarchy of thermodynamic response functions. A universal covariance identity connects derivatives of tempered expectations to posterior fluctuations, placing WAIC, WBIC, and singular fluctuation within a single unified response framework. The results suggest that thermodynamic response theory offers a natural organizing framework for interpreting model complexity, predictive variability, and structural reorganization in singular Bayesian learning, bridging statistical learning theory with statistical mechanics.

Paper Overview

Field: Machine Learning
Authors: Anonymous
Posted: 2026-03-06
arXiv: 2603.05500

Abstract

Singular statistical models—including mixtures, matrix factorization, and neural networks—violate regular asymptotics due to parameter non-identifiability and degenerate Fisher geometry. We show that posterior tempering induces a one-parameter deformation of the posterior distribution whose associated observables generate a hierarchy of thermodynamic response functions. A universal covariance identity links derivatives of tempered expectations to posterior fluctuations, placing WAIC, WBIC, and singular fluctuation within a unified response framework. Our results suggest that thermodynamic response theory provides a natural organizing framework for interpreting complexity, predictive variability, and structural reorganization in singular Bayesian learning.

Key Points

Singular models (mixtures, matrix factorization, neural networks) break the standard regular asymptotic assumptions of classical statistics.
Posterior tempering is treated as a one-parameter deformation of the posterior, analogous to temperature in statistical mechanics.
Derivatives of tempered expectations yield a hierarchy of thermodynamic response functions.
A universal covariance identity connects these response functions to posterior fluctuations.
WAIC, WBIC, and singular fluctuation are unified within this thermodynamic response framework, offering a common lens on complexity, predictive variability, and structural reorganization in singular Bayesian learning.