You train a variational autoencoder (VAE). You feed it a picture of a cat, it compresses the image into a latent vector, and reconstructs the image. Training finishes, and the reconstruction quality looks good.
But then you inspect the latent vector — and you find that most of its dimensions are always zero. Or the learned encoder distribution matches the prior exactly — carrying no information at all. The model can reconstruct the cat, but it's not doing so by "understanding what a cat looks like." It's mostly copying information from input straight to output, and the latent variables are just decoration.
This is called posterior collapse, one of the most stubborn failure modes of VAEs. People have known about it for a decade, but there's still no clean solution.
Aliahmadi, Babayan, and van Anders recently posted a paper on arXiv (2605.16164) proposing an approach that, after reading it, feels like it should have been obvious all along: get rid of the latent prior entirely.
🧩 Why posterior collapse happens
The VAE objective has two parts. One is the reconstruction loss — the decoder's output should look as much like the input as possible. The other is the KL divergence — the encoder's output distribution should stay close to the prior distribution.
The root of the problem is that KL term. It tells the encoder: "Your latent distribution should be close to a standard Gaussian."
This creates tension. The reconstruction loss wants the latents to carry as much information as possible — ideally every detail of every training sample. The KL divergence wants the latents to *forget* — to be pulled toward a standard Gaussian that remembers nothing.
If the KL term is too strong — or the decoder is too strong — the encoder gives up. It says: fine, I won't learn anything. I'll pass all the information from the input directly to the decoder and keep nothing in the latent variables. The latent space becomes a dead end. That's posterior collapse.
Once collapsed, your VAE degenerates into a deterministic autoencoder, and the latent variables — which you hoped would capture the core structure of the data — are empty.
🔧 Their fix: stop pushing, let entropy do the work
The core idea of the EAE — Entropic AutoEncoder — is direct: remove the KL term. No explicit prior constraint.
But this creates a serious problem: without a prior constraint, the latent distribution can inflate arbitrarily. If the encoder learns an arbitrarily complex distribution, what makes the latents meaningful?
Their answer: instead of one encoder, use an ensemble of encoders. Each encoder in the ensemble has a slightly different initialization, and they co-evolve through a free-energy minimization process. The ensemble itself — not via KL divergence, but via the statistical structure among ensemble members — imposes an implicit constraint on the latent variables.
The core mechanism of this constraint is the maximum entropy principle. In nature, when a system has multiple equally good solutions, the realized system tends to favor the region with the largest "volume" — the maximum-entropy region. What EAE's ensemble encoders do is find these high-entropy regions. The decoder, on the other side, searches for latent variables that are genuinely informative for reconstruction.
The tug-of-war between the two sides is not an explicit adversarial setup of "KL divergence vs. reconstruction loss," but a collaborative exploration: "the ensemble finds high-entropy regions, the decoder finds useful information."
📸 What they observed
- Reaction-diffusion dynamics: EAE learned the known superposition of low-dimensional dynamics — the latents accurately captured the system's physical degrees of freedom.
- MNIST: EAE's latents automatically discovered implicit class boundaries between digits — not hinted by any supervision signal, but grown out of the reconstruction constraint itself.
- CelebA faces: EAE showed a hierarchical understanding. Some global latent dimensions encode "this is a human face" — attributes shared by all faces. Other dimensions encode "this is *this particular* face" — individual features. From "all people" to "this person," a clear hierarchy emerges — all from an autoencoder with no explicit prior constraint.
🤷 What I don't know yet
There are things I'm not sure about.
First, how does EAE's multi-level latent structure ("face" first, then "person") emerge naturally from the reconstruction loss? In VAEs, the KL term acts somewhat as an information bottleneck — limiting the latent information capacity and forcing the latents to pick the most important information. EAE removes this bottleneck, replacing it with an implicit constraint based on ensemble entropy. I can intuit the general direction of this mechanism, but I don't fully grasp the quantitative relationship between the implicit constraint and the explicit bottleneck.
Second, how many forward passes does EAE's encoder ensemble require in practice? Does each data point need to be run through all ensemble members to find the optimal encoding? If the ensemble is large — say dozens or hundreds of encoders — the compute cost might make EAE impractical at scale. The paper validates on MNIST and CelebA but doesn't discuss the computational overhead of scaling to ImageNet-sized data.
Third, does the "implicit prior" always produce meaningful structure, or does it merely relocate posterior collapse from "empty latents" to "redundant latents"? If the decoder is still too strong, could EAE's latents also degenerate into a high-entropy but useless representation? The paper's theory discusses how free-energy minimization prevents this, but gives no explicit sufficient conditions.
Still, the core idea is elegant: when the explicit KL constraint conflicts with the encoder's objective, don't push, don't pull. Let the natural tendency toward maximum entropy implicitly shape the representation. Don't tell the latent variables what they "should look like" — let them discover what they should become through exploration.
---
References
1. Aliahmadi, H., Babayan, I., & van Anders, G. (2026). *Entropic Auto-Encoding via Implicit Free-Energy Minimization*. arXiv:2605.16164 [cs.LG]. https://arxiv.org/abs/2605.16164 2. Kingma, D. P., & Welling, M. (2014). *Auto-Encoding Variational Bayes*. ICLR 2014. 3. Bowman, S. R., et al. (2016). *Generating Sentences from a Continuous Latent Space*. CoNLL 2016. 4. Razavi, A., et al. (2019). *Generating Diverse High-Fidelity Images with VQ-VAE-2*. NeurIPS 2019. 5. Jaynes, E. T. (1957). *Information Theory and Statistical Mechanics*. Physical Review, 106(4), 620-630.