You train a variational autoencoder (VAE). You feed it a picture of a cat, it compresses the image into a latent vector, and reconstructs the picture. Training finishes, and the reconstruction quality looks good.
But then you inspect the latent vector—and you find that most of its dimensions are always zero. Or, equivalently, the learned encoder distribution is identical to the prior—it contains no information at all. The model can reconstruct the cat, but not because it "understood what a cat looks like and reconstructed it." Instead, it copied most of the information straight from input to output; the latent variable is just decoration.
This is called posterior collapse, one of the most stubborn failure modes of VAEs. It was known ten years ago, yet there is still no clean solution.
Aliahmadi, Babayan, and van Anders recently posted a paper on arXiv (2605.16164) proposing an approach that, after reading it, feels like it should have been obvious much earlier: get rid of the prior on the latent variable entirely.
🧩 Why posterior collapse happens
The VAE objective has two parts. One is reconstruction loss—you want the decoder's output to match the input as closely as possible. The other is the KL divergence—you want the encoder's output distribution to stay close to the prior distribution.
The root of the problem is that KL term. It tells the encoder: "Your latent distribution should be close enough to a standard Gaussian."
This creates a tension. The reconstruction loss wants the latent variable to carry as much information as possible—the encoder would ideally memorize every detail of every training sample. The KL divergence wants the latent variable to *forget* information—pulling it toward a standard Gaussian that remembers nothing.
If the KL term is too strong—or the decoder is too powerful—the encoder gives up. It says: fine, I won't learn anything. I'll just pass all the information from the input directly to the decoder and keep nothing in the latent variable. The latent variable becomes a dead end. That is posterior collapse.
Once collapsed, your VAE degenerates into a deterministic autoencoder, and the latent variable—the core representation you hoped would capture the structure of the data—is empty.
🔧 Their fix: stop pushing, let entropy do the work
The core idea of EAE—the Entropic AutoEncoder—is simple: remove the KL term. No explicit prior constraint.
But this raises a serious problem: without a prior constraint, the latent distribution can inflate arbitrarily. The encoder could learn arbitrarily complex distributions—so how could the representation remain meaningful?
Their solution: instead of one encoder, use an ensemble of encoders. Each encoder in the ensemble has a slightly different initialization, and they co-evolve through a free-energy minimization process. The ensemble itself—not via KL divergence, but via the statistical structure among ensemble members—imposes an implicit constraint on the latent variable.
The central mechanism of this constraint is the maximum entropy principle. In nature, when a system has multiple "equally good" solutions, the actual system tends to favor the region with the largest "volume"—the region of maximum entropy. What EAE's ensemble of encoders does is find such high-entropy regions. The decoder, on the other side, searches for the informative latent variables that genuinely help reconstruction.
The push-and-pull between the two sides is not an explicit adversarial game of "KL divergence vs. reconstruction loss," but a collaborative exploration: the ensemble seeks high-entropy regions, and the decoder seeks useful information.
📸 What they saw
- On data from reaction-diffusion dynamics, EAE learned the known superposition of low-dimensional dynamics—the latent variables accurately captured the system's physical degrees of freedom.
- On MNIST, EAE's latent variables automatically discovered implicit class boundaries between digits. Not prompted by any supervised signal—these boundaries emerged on their own under the reconstruction constraint.
- On the CelebA face dataset, EAE showed a hierarchical understanding. Some global latent dimensions encode "this is a human face"—attributes shared by all faces. Other dimensions encode "this is *this particular* face"—individual features. From "all people" to "this person," a clear hierarchy emerges. And all of it comes from an autoencoder with no explicit prior constraint.
🤷 What I don't know
There are things I'm uncertain about.
First, how does EAE's multi-level latent structure ("face" first, then "person") emerge naturally from the reconstruction loss? In a VAE, the KL term acts to some extent as an information bottleneck—limiting the information capacity of the latent variable and forcing it to pick out the most important information. EAE removes this bottleneck but uses the ensemble encoders' entropy as an implicit constraint. I can intuit the general direction of this mechanism, but I haven't fully grasped the quantitative relationship between implicit constraints and explicit bottlenecks.
Second, how many forward passes does EAE's ensemble of encoders require in practice? Does every data point need to be run through all ensemble members to find the optimal encoding? If the ensemble is large—say, dozens or hundreds of encoders—the computational cost could make EAE impractical on large-scale datasets. The paper validates on MNIST and CelebA, but doesn't discuss the computational overhead of scaling to ImageNet-scale data.
Third, does the "implicit prior" always produce meaningful structure, or does it just move posterior collapse from "empty latent variables" to "redundant latent variables"? If the decoder is still too strong, could EAE's latent variables also degenerate into a high-entropy but useless representation? The paper's theoretical section discusses how free-energy minimization prevents this, but gives no explicit sufficient conditions.
Still, the core idea is beautiful: when the explicit KL constraint conflicts with the encoder's objective, don't push, don't pull. Let the natural tendency toward entropy maximization shape the representation implicitly. Don't tell the latent variable what it "should look like"—let it find out for itself through exploration.
---
References
1. Aliahmadi, H., Babayan, I., & van Anders, G. (2026). *Entropic Auto-Encoding via Implicit Free-Energy Minimization*. arXiv:2605.16164 [cs.LG]. https://arxiv.org/abs/2605.16164 2. Kingma, D. P., & Welling, M. (2014). *Auto-Encoding Variational Bayes*. ICLR 2014. 3. Bowman, S. R., et al. (2016). *Generating Sentences from a Continuous Latent Space*. CoNLL 2016. 4. Razavi, A., et al. (2019). *Generating Diverse High-Fidelity Images with VQ-VAE-2*. NeurIPS 2019. 5. Jaynes, E. T. (1957). *Information Theory and Statistical Mechanics*. Physical Review, 106(4), 620-630.