Paper Slam 4/26: When Information Theory Meets Geometric Symmetry — A Deep Dialogue Between Two Papers
> arXiv:2604.21849 | arXiv:2604.21809
Introduction: Two Worlds, One Anxiety
In April 2026, two seemingly unrelated arXiv papers appeared almost simultaneously. One shows how to replace KL divergence with Integral Probability Metrics (IPMs) for Bayesian optimal experimental design (BOED); the other builds diffusion models directly on quotient spaces to generate molecular structures. One comes from the University of Maryland's mathematics department; the other from Peking University and Microsoft Research Asia. One targets experiment planning in scientific computing, the other structure generation in computational chemistry.
Yet read side by side, they share a deep anxiety: are our standard mathematical tools deceiving us when facing real-world complexity?
Paper one (Wu, Liang, Yang) points out that KL divergence — the "standard answer" for measuring distributional distance — is extremely fragile in the tails: a tiny surrogate-model error in a low-probability region makes KL explode logarithmically. Paper two (Xu, Wang, Luo et al.) finds that conventional group-equivariant diffusion models guarantee symmetric invariance of the output distribution, but the network is still forced to learn redundant motion inside equivalence classes — like asking a painter not only to paint a portrait but to precisely specify the canvas's angle in the room.
The solutions differ, but point the same direction: don't patch the problem's surface — return to its geometric essence. Paper one says: stop measuring distribution differences by density ratios; use differences of test-function expectations instead. Paper two says: stop forcing invariance in the original space; model directly on the quotient space. One is an information-theoretic view, the other a differential-geometric one, but the core insight is identical: find what truly needs to be learned, and find a way not to learn everything else.
---
Chapter 1: IPM-based BOED — When KL Divergence Fails
1.1 The Problem
Optimal Experimental Design (OED) asks: given a limited experimental budget, which experimental conditions minimize uncertainty about unknown parameters? Bayesian OED formalizes this with a prior p(x), a design ξ, an observation y, and a posterior p(x|y,ξ). The classic Expected Information Gain (EIG) uses KL divergence from posterior to prior:
EIG(ξ) = ∫_𝒴 ∫_𝒳 log(p(x|y,ξ)/p(x)) p(x|y,ξ) p(y|ξ) dx dy
1.2 Three Fatal Weaknesses of KL
1. Support mismatch: KL requires overlapping supports (absolute continuity). With a continuous prior and a discrete Monte Carlo approximation, KL is technically infinite. 2. Tail underestimation: KL is sensitive to density ratios. If the surrogate gives near-zero likelihood in a low-probability region where the true model is non-negligible, KL explodes. The authors' rare-event contamination experiment shows KL estimates locked to contamination-sample counts with huge variance, while Wasserstein, Energy Distance, and MMD respond far more smoothly. 3. Nested expectation computational nightmare: EIG involves nested integrals; even Foster et al.'s variational methods keep the log-density-ratio objective and inherit these structural problems.
1.3 IPMs: From Density Ratios to Expectation Differences
An IPM asks: to what extent do two distributions give different expectations for the same class of test functions?
γ_ℱ(P,Q) = sup_{f∈ℱ} |E_X∼P[f(X)] - E_Y∼Q[f(Y)]|
Choosing different ℱ yields different metrics:
- Lipschitz functions (‖f‖_L ≤ 1) → 1-Wasserstein distance
- RKHS unit ball → Maximum Mean Discrepancy (MMD)
- Semimetric-of-negative-type functions → Energy Distance (ED)
- Bounded IPMs (e.g., MMD with translation-invariant radial kernels): test functions are globally bounded (sup|f| ≤ M). Utility error is unconditionally controlled linearly by L1 likelihood error — no tail-decay or global-Lipschitz assumptions needed.
- Unbounded IPMs (Energy Distance grows as O(‖x‖^{1/2}), W1 as O(‖x‖)): stronger geometric power — capturing spatial shifts — but stability requires sub-Gaussian priors and globally Lipschitz forward models.
- Data augmentation: randomly apply SE(3) transforms during training.
- Equivariant architectures (e.g., EGNN): guarantee D_θ(g·x, t) = g·D_θ(x, t).
- GeoDiff: aligns targets to the noisy input's frame; removes variance but the optimum is still E[A_𝐱_t(𝐱_1)|𝐱_t], which depends on 𝐱_t's direction.
- AF3: aligns targets to model output, removing all equivariant learning, but outputs are arbitrary along the group — the sampler cannot recover the correct target distribution.
- Boltz-1: aligning predictions at each sampling step degenerates to GeoDiff-like behavior.
Crucially, IPMs need no densities — all are estimable directly from samples. The paper defines the IPM-based utility:
U_ℱ(ξ) = ∫_𝒴 γ_ℱ(p(x), p(x|y,ξ)) p(y|ξ) dy
1.4 Stability Theory: Bounded vs. Unbounded
The paper's core theoretical contribution answers: if the surrogate model has errors, or the prior is only approximate, how much does the design utility degrade?
KL's stability cannot fit this framework: its test function (the log density ratio) has uncontrolled growth and responds to multiplicative rather than additive error — a small L∞ perturbation can make KL explode.
1.5 Plug-and-Play Computation
IPMs are naturally sample-based, so the BOED pipeline stays modular. The paper gives a closed-form W1 expected utility for A/B testing, shows IPM-based utilities have wider near-optimal regions and more stable optimization landscapes in preference learning, and plugs a neural optimal transport estimator (OT-ICNN) into the same framework in high dimensions. The methodological signal: the boundary between information theory and optimal transport is blurring — when what matters is "how distributional differences affect decisions," geometric metrics can outperform information-theoretic elegance.
---
Chapter 2: Quotient-Space Diffusion — Letting Symmetry Disappear
2.1 The Coordinate Dilemma of Molecules
A molecule with N atoms is a 3N-dimensional vector in ℝ^{3N}. But rotating or translating the whole molecule doesn't change its chemistry — SE(3) maps many vectors to "the same molecule." Traditional fixes:
Both make the *distribution* invariant but not the *learning process*: the network still learns motion inside each equivalence class. The canvas's angle was never part of the portrait.
2.2 The Quotient Space
Instead of learning an SE(3)-invariant distribution on ℝ^{3N}, build the diffusion model on the quotient space ℝ^{3N}/SE(3), whose points are equivalence classes. Rotation and translation simply don't exist there — nothing to learn.
Technically the quotient is a Riemannian manifold. The paper's Theorem 1: the projection of a G-equivariant diffusion on ℳ to 𝒬 = ℳ/𝒢 is itself a diffusion, satisfying a corrected SDE:
d𝐲_t = ((π_* 𝐛_t)(𝐲_t) - σ_t²/2 𝐡(𝐲_t)) dt + σ_t d𝛚_t
where 𝐡 is the mean curvature vector field — a correction from quotient geometry reflecting how equivalence-class volume changes along the space.
2.3 Horizontal Lift: Back to Computable Ground
Simulating on the quotient is hard (no global coordinates). The solution: horizontal lift. The vertical space 𝒱_𝐱 = Ker π_*𝐱 corresponds to directions within the equivalence class (rotations/translations); the horizontal space ℋ_𝐱 is its orthogonal complement — true deformation directions. Keeping only horizontal components leaves the quotient projection unchanged while shortening sampling paths.
For SE(3), the projection operator P is explicit. For the SO(3) rotation part:
P_𝐱(𝐯) = (𝐯^(n) - 𝐊⁻¹(∑ x^(n') × v^(n')) × x^(n))_n, 𝐊 = ∑ ‖x^(n)‖² 𝐈 - ∑ x^(n) x^(n)ᵀ
Physically: it removes total angular momentum — horizontal vectors are "pure deformation," changing internal structure without rotating the molecule.
2.4 Redesigned Training Objective
ℒ(θ) = 𝔼 ‖P_𝐱_t (D_θ(𝐱_t, t) - 𝐱_1)‖²
For any vertical 𝐯^𝒱, D_θ + 𝐯^𝒱 has identical loss: the model need not learn within-class predictions at all. Comparing four strategies:
| Strategy | Removes equivariant DOF learning | Removes equivariant DOF variance | Sampling compatibility | |---|---|---|---| | Traditional loss | ✗ | ✗ | ✓ | | GeoDiff alignment | ✗ | ✓ | ✗ | | AF3 alignment | ✓ | ✓ | ✗ | | Quotient-space diffusion | ✓ | ✓ | ✓ |
2.5 Why Alignment Fails
2.6 Experiments
Small molecules (GEOM-QM9, GEOM-DRUGS): 9%–23% relative improvement over ET-Flow. On GEOM-QM9, Coverage recall rises from 95.98% to 96.40%; AMR mean drops from 0.076 to 0.069. Notably, AF3 alignment *regresses* recall to 92.67% — direct evidence of the train-sample mismatch's damage.
Proteins (Proteína baseline): a 60M-parameter quotient-space model beats the 200M baseline on key distribution metrics. SDE sampling: designability improves from 96.0% to 97.6% at γ=0.35. ODE sampling: FPSD vs. PDB drops from 83.2 to 69.9; fJSD vs. PDB from 21.9 to 17.6. Against AF3 alignment: designability collapses from 13.8% to 3.8%.
---
Chapter 3: A Dialogue of Two Geometry-Aware Approaches
3.1 Shared Problem Structure
Both papers address the same meta-problem: some degrees of freedom are unnecessary, but standard tools force you to handle them. In BOED, it's dependence on density ratios; in molecular diffusion, it's motion inside equivalence classes. Both change the problem's space rather than patch it: measure space with IPMs, quotient space for diffusion.
3.2 Metric vs. Space
IPMs perform geometry in function space (Lipschitz → spatial displacement; RKHS → smooth feature differences). Quotient diffusion performs geometry in configuration space (orthogonal decomposition into vertical symmetry directions and horizontal physical directions). Both embody one principle: decompose structure into "changing" and "invariant" parts, then handle only the changing part.
3.3 Two Sides of Stability
Paper one's stability is *utility stability under approximation*: bounded IPMs give unconditional linear control via additive error; unbounded IPMs need tail conditions. Paper two's stability is *distribution recovery under symmetry*: Corollary 3 proves the lifted process and original process yield identical distributions — an exact result via geometric projection. Together: stability can come from an insensitive metric or from a process confined to insensitive directions.
3.4 Low-Density Regions and Low-Dimensional Structures
Paper one tackles low-density tails (KL explodes at small denominators; IPMs don't). Paper two exploits low-dimensional structure (ℝ^{3N}/SE(3) has 3N−6 dimensions — shorter sampling paths, smaller learning space). Shared theme: complexity often comes from redundancy in the problem formulation, not the problem itself.
3.5 Two Kinds of Plug-and-Play
Paper one is modular at the metric level (the template accepts any sample-estimable discrepancy, including neural OT). Paper two is flexible at the architecture level (the projection operator modifies only the objective and sampler). Both offer principles, not recipes.
---
Chapter 4: Deeper Reflections
4.1 From KL Hegemony to Geometric Pluralism
KL dominates BOED by historical inertia, not correctness. The paper's deeper contribution: turning metric choice from dogma into a design decision, based on error structure, tail behavior, and computational constraints.
4.2 The Cost of Symmetry
Symmetry should be understood as reduction, not constraint. Equivariant networks constrain programs with type checks; quotient-space methods design a language without such errors — working in a space where symmetry has already been "divided out."
4.3 What Theoretical Guarantees Really Mean
Paper one's bounds (Theorems 4.1, 4.2) are potentially loose upper bounds — their value is qualitative insight for tool selection. Paper two's guarantee (Corollary 3) is exact but assumes precise projection computation. Guarantees don't "prove methods work"; they reveal why they work, telling you which conditions must hold in new settings.
4.4 General-to-Specific Architecture
Both papers build general frameworks (IPMs over W1/MMD/ED; Riemannian quotients over general Lie groups), then specialize — signaling potential beyond current applications: any experiment-design scenario comparing prior and posterior; crystal structure generation, molecular dynamics, and physics-field parameterization for the quotient approach.
4.5 Reproducibility Caveats
Paper two candidly notes reproduced ET-Flow results "do not exactly match" the original due to data-pipeline changes, and Foldseek bugs limited some protein comparisons. Honest disclosure builds trust but reminds us: reproducibility depends on pipelines, evaluation tools, and versions, not just methods.
4.6 Shared Limitations
Paper one assumes definable priors and likelihoods; systematic (directional) forward-model errors may exceed the guarantees. Paper two assumes free, proper, isometric group actions, requiring exclusion of degenerate (collinear) configurations; numerically, 𝐊 may approach singularity. Neither handles partial symmetry breaking or configuration-dependent symmetry groups.
---
Conclusion: Back to Feynman's Question
"What I cannot create, I do not understand." Both papers help us *create* — not by adding layers, parameters, or data, but by returning to the mathematical essence and removing what should never have existed. Paper one removes dependence on density ratios; paper two removes the learning of motion inside equivalence classes. Both show: understanding a problem's true structure matters more than approximating it with more powerful tools.
IPM and quotient-space diffusion won't become universal answers, but they remind us: when standard tools feel wrong, perhaps the tools themselves need reinventing. Feynman would have liked these papers — not for their complex mathematics, but for their simple question: "Can we do this in a simpler way?" The answer, after many pages of proofs, is: yes.
---
Reference Quick View
Paper one key citations: Lindley (1956), Müller (1997), Gretton et al. (2009, 2012), Foster et al. (2019), Helin et al. (2025), Amos et al. (2017)
Paper two key citations: Ho et al. (2020), Song et al. (2021), Albergo et al. (2023), Xu et al. (2022) [GeoDiff], Hoogeboom et al. (2022) [EDM], Abramson et al. (2024) [AlphaFold 3], Geffner et al. (2025) [Proteína], Lee (2018)
*Generated by the Paper Slam 4/26 project. All technical details defer to the original papers.*