Paper Slam 4/26: When Information Theory Meets Geometric Symmetry — A Deep Dialogue Between Two Papers

Forum topic · 小凯 · 2026-04-28

Summary

This forum post compares two arXiv papers (arXiv:2604.21849 and arXiv:2604.21809) that tackle redundant mathematical structure from different angles. The first, from University of Maryland mathematicians, replaces KL divergence in Bayesian optimal experimental design (BOED) with Integral Probability Metrics (IPMs) such as Wasserstein distance, MMD, and Energy Distance. It proves stability guarantees: bounded IPMs control utility error unconditionally under surrogate-model L1 error, while unbounded IPMs need tail assumptions; KL divergence, by contrast, can explode under small perturbations in low-density regions. The second, from Peking University and Microsoft Research Asia, builds diffusion models directly on the quotient space R^{3N}/SE(3) for molecular structure generation. Using horizontal lifts and an explicit projection operator that removes net rotation/translation, the model eliminates redundant equivariant degrees of freedom while provably recovering the target distribution (Corollary 3). Experiments show 9-23% relative gains on GEOM-QM9/DRUGS with ET-Flow, and a 60M-parameter quotient-space model beating a 200M-parameter Proteina baseline, while AF3-style alignment can degrade designability severely. The post argues both papers share one principle: identify what truly needs to be learned and stop learning everything else.

Overview

A Chinese forum post (Paper Slam 4/26) offers a side-by-side deep read of two April 2026 arXiv papers that, despite different domains, share a common theme: remove redundant structure from the problem formulation instead of patching over it.

Paper 1 (arXiv:2604.21849, University of Maryland): replaces KL divergence with Integral Probability Metrics (IPMs) for Bayesian Optimal Experimental Design (BOED).
Paper 2 (arXiv:2604.21809, Peking University & Microsoft Research Asia): builds diffusion models directly on the quotient space R^{3N}/SE(3) for molecular structure generation.

Key points

Paper 1: IPM-based BOED

Why KL fails: (1) support mismatch makes KL infinite for discrete Monte Carlo priors; (2) tail underestimation — small surrogate errors in low-probability regions make the log density ratio explode; (3) nested expectations make EIG computation hard, and variational methods still rely on log density ratios.
IPM framework: γ_ℱ(P,Q) = sup over test functions f∈ℱ of |E_P[f] − E_Q[f]|. Choosing Lipschitz functions yields 1-Wasserstein, RKHS unit ball yields MMD, semimetrics of negative type yield Energy Distance. IPMs are density-free and sample-estimable.
Stability theory: bounded IPMs (e.g., MMD) have utility error linearly controlled by L1 likelihood error *unconditionally*; unbounded IPMs (ED grows as O(‖x‖^{1/2}), W1 as O(‖x‖)) need sub-Gaussian priors and globally Lipschitz forward models. KL's test function (log density ratio) has uncontrolled growth and is sensitive to multiplicative error.
Plug-and-play computation: closed-form W1 utility for A/B testing, stable optimization landscapes in preference learning, and compatibility with neural optimal transport estimators (OT-ICNN).

Paper 2: Quotient-space diffusion

The problem: rotating or translating a molecule does not change it chemically, yet ℝ^{3N} coordinates force networks to learn motions inside each SE(3) equivalence class. Data augmentation and equivariant architectures (e.g., EGNN) make the *distribution* invariant but not the learning process.
Core theorem: projecting a G-equivariant diffusion on ℳ onto the quotient 𝒬 = ℳ/𝒢 yields another diffusion with a corrected SDE involving the mean curvature vector field.
Horizontal lift: decompose tangent vectors into vertical (group orbits) and horizontal (true deformations) components; an explicit projection operator P_x removes the system's total angular momentum for SE(3). Training only needs the horizontal component of the prediction error, so vertical outputs are all optimal.
Comparison table: only quotient-space diffusion simultaneously removes equivariant degrees of freedom, removes their variance, and preserves sampling guarantees. GeoDiff-style alignment leaves residual learning burden; AF3-style alignment breaks the sampler's distributional guarantees.
Experiments: 9%–23% relative improvements with ET-Flow on GEOM-QM9/GEOM-DRUGS (Coverage recall 95.98%→96.40%, AMR 0.076→0.069); AF3 alignment *drops* recall to 92.67%. On proteins, a 60M-parameter quotient model beats the 200M Proteina baseline (designability 96.0%→97.6%; FPSD vs. PDB 83.2→69.9); AF3 alignment collapses designability from 13.8% to 3.8%.

Shared insights

Both change the problem's space rather than adding constraints: Paper 1 works directly in measure space via IPMs; Paper 2 works in the quotient manifold (3N−6 dimensions instead of 3N).
Both exploit orthogonal decompositions — IPMs project distribution differences onto a test-function class; quotient diffusion splits tangent spaces into vertical/horizontal parts.
Stability comes either from insensitive metrics (bounded test functions) or from restricting motion to insensitive directions (horizontal projection).
Complexity often comes from the problem *formulation*, not the problem itself: low-density tails in one case, redundant symmetry directions in the other.

Limitations noted

Paper 1 assumes well-defined priors/likelihoods and cannot handle directional (systematic) model errors; Paper 2 assumes free, proper, isometric group actions and degenerate collinear configurations must be excluded.
Neither addresses partial symmetry breaking or configuration-dependent symmetry groups.
Reproducibility caveats: the second paper's reproduced ET-Flow results differ slightly due to data-pipeline changes, and a known Foldseek bug limited protein comparison analyses.

Conclusion

Both papers embody the principle: find what truly needs to be learned, then do not learn anything else. Quotient-space diffusion treats symmetry as a reduction, not a constraint; IPM-based BOED turns metric choice from dogma into a design decision grounded in error structure.

References mentioned

Paper 1: Lindley (1956), Müller (1997), Gretton et al. (2009, 2012), Foster et al. (2019), Helin et al. (2025), Amos et al. (2017).

Paper 2: Ho et al. (2020), Song et al. (2021), Albergo et al. (2023), Xu et al. (2022, GeoDiff), Hoogeboom et al. (2022, EDM), Abramson et al. (2024, AlphaFold 3), Geffner et al. (2025, Proteína), Lee (2018).

> Original arXiv IDs: arXiv:2604.21849, arXiv:2604.21809. All technical details per the original papers.