Forgetting Is Not Erasure: Shallow vs. Deep Forgetting, and a Single Attention Layer Unlocks Image Generation

Forum topic · ✨步子哥 · 2026-01-10

Summary

A December 2025 Chinese forum post analyzes two arXiv papers that challenge assumptions about neural network memory and generation. ETH Zurich researchers distinguish shallow forgetting (output-level accuracy loss driven by classifier misalignment and statistical artifacts) from deep forgetting (feature-level degradation). They show old features are not erased but drift into subspaces orthogonal to active ones, decaying exponentially under weight decay. Remarkably, an infinitesimal replay buffer suffices to anchor feature geometry (SNR scaling as Theta(r^2)), while repairing shallow forgetting requires large buffers with O(b^-1/2) estimation error—validated on CIFAR-100 and MiniImageNet. Separately, Apple's Feature Auto-Encoder (FAE) shows a single self-attention layer plus linear projection can adapt frozen visual encoders (DINOv2, SigLIP) into generative engines, achieving FID 1.29 on ImageNet 256x256 with CFG and 7-13x faster convergence than SD-VAE. Both works point toward modular, efficient AI—lightweight adapters over frozen universal backbones—as scaling laws bend.

Forgetting Is Not Erasure: Shallow vs. Deep Forgetting, and a Single Attention Layer Unlocks Generation

*Translation and summary of a zhichai.net forum post discussing two December 2025 arXiv papers.*

Imagine a neural network theater: actors (model parameters) who performed brilliantly in Act I suddenly forget their lines when Act II begins. This is catastrophic forgetting—a longstanding pain point in AI. But two papers appearing almost simultaneously on arXiv in December 2025 illuminate the stage from a new angle: forgetting is not erasure but clever "hiding," and imaginative feats like image generation require only a thin attention curtain to awaken frozen pretrained actors.

The ETH Zurich team rigorously divides forgetting into "shallow" and "deep" varieties, while Apple's research uses a minimalist Feature Auto-Encoder (FAE) to prove a single attention layer suffices to turn a visual encoder that "understands the world" into a generative engine that "creates" it. Both point the same direction: less is more—simple mechanisms can solve problems once thought to require massive parameter counts.

Shallow vs. Deep Forgetting: Makeup Fades, Memory Remains

When a network learns task B after task A, performance on A typically collapses. Researchers long believed old knowledge was overwritten like a repeatedly wiped blackboard.

ETH Zurich found otherwise, decomposing forgetting into two kinds:

Shallow forgetting occurs at the output level: classifier-head accuracy on old tasks crashes. Formally:

F^{\text{shallow}}_{i \to j} = A_{jj} - A_{ij}

where $A_{jj}$ is current-task accuracy and $A_{ij}$ is accuracy on task i after learning task j. Its culprits are classifier misalignment and statistical artifacts (missing covariance, mean-norm inflation). Repairing it requires substantial old data.

Deep forgetting occurs at the feature level, measured by frozen linear probes:

F^{\text{deep}}_{i \to j} = A^*_{jj} - A^*_{ij}

Surprisingly, even when outputs are scrambled, features often remain well linearly separable. The model has not truly erased old memories—it has hidden them in regions orthogonal to the currently active subspace.

> Neural Collapse: a phenomenon where same-class features collapse toward class means, and different class means form a simplex ETF (equiangular tight frame) geometry. This structure enhances linear separability and provides a mathematical handle for analyzing forgetting.

Like childhood toys stowed in an attic—dusty and displaced (shallow forgetting) but still in place (deep memory intact)—geometric drift proceeds without replay data as old-task samples decay exponentially in orthogonal directions:

\phi_{t,S^\perp}(x) = (1 - \eta\lambda)^{t - t_0} \phi_{t_0,S^\perp}(x)

where $\eta$ is the learning rate and $\lambda$ the weight decay. Old samples thus become out-of-distribution (OOD), unifying continual learning with OOD detection.

The Asymmetric Miracle of Tiny Replay Buffers

If deep memory is merely hidden, a little old data should bring it back. The answer is yes—with striking efficiency. The researchers found a replay efficiency gap:

Preventing deep forgetting requires only an infinitesimal (nonzero) replay ratio $$r > 0$$ to anchor feature geometry, keeping linear separability SNR from decaying:

\lim_{t\to\infty} \text{SNR}(c_1, c_2) \in \Theta(r^2)

Repairing shallow forgetting requires far larger buffers, since small buffers introduce statistical bias with estimation error converging as $O(b^{-1/2})$ .

Experiments on CIFAR-100 and MiniImageNet confirmed this: tiny buffers nearly eliminate deep forgetting, while shallow forgetting still needs more data.

| Dimension | Shallow Forgetting | Deep Forgetting | |---|---|---| | Definition | Output-level accuracy drop | Feature-level linear separability drop | | Measurement | Direct accuracy $A_{ij}$ | Frozen linear-probe accuracy $A^*_{ij}$ | | Main cause | Classifier misalignment, statistical artifacts | Geometric drift into orthogonal subspaces | | Replay need | Large buffers for correction | Infinitesimal amounts anchor geometry | | Implication | Costly repair, traditional bottleneck | Efficient continual learning possible |

Single-Head vs. Multi-Head

In single-head (class-incremental) settings, replay induces global Neural Collapse with feature rank up to $$nK - 1$$ . In multi-head (task-incremental) settings, each task forms local NC with mutually orthogonal subspaces, rank-capped at $$n(K-1)$$ . Modular design thus naturally reduces interference.

Apple's FAE: One Attention Layer Lights Up Generation

Traditional generative models train complex VAEs from scratch to compress images into latent spaces. Apple asked: given powerful frozen visual encoders (DINOv2, SigLIP) with excellent semantic geometry, why not use them directly with a minimal interface?

FAE's architecture is strikingly simple: a single self-attention layer plus linear projections de-redundizes patch embeddings, followed by dual decoders:

Feature decoder: a Transformer reconstructs original embeddings with L2 loss + KL regularization
Pixel decoder: generates images with adversarial, perceptual, and reconstruction losses

Why is a single layer enough? Because adaptation strength should be far lower than pretraining—too many parameters overfit and distort original semantic geometry. Weak adaptation stays close to the original representations, preserving patch similarity and cross-image matching.

| Benchmark | FAE FID (no CFG) | FAE FID (CFG) | vs. Baseline | Training Efficiency | |---|---|---|---|---| | ImageNet 256×256 (800 epochs) | 1.48 | 1.29 | Beats SD-VAE | Standard | | ImageNet 256×256 (80 epochs) | 2.08 | 1.70 | 7–13× faster convergence | High | | MS-COCO (CC12M data) | 7.47 | 6.90 | Matches with less data | Modular reuse | | STARFlow (flow matching) | 2.67 | – | Outperforms SD-VAE | Stable dynamics |

A low-dimensional latent space (32 dims) stabilizes diffusion, and tricks like Gaussian-noise pretraining and time-shifting let FAE match or exceed complex VAEs in short training times.

From Brute Force to Elegant Efficiency

Scaling laws have guided AI progress for five years, but the line is bending. Ilya Sutskever has said scaling is "peaking," with power, chips, and data barriers expected around 2030; Harvard's precision-scaling work shows diminishing marginal returns.

These two papers offer an alternative path: minimal interventions (tiny replay, single attention layer) solving the hardest bottlenecks (continual learning, understanding-to-generation). Together they sketch a modular picture of "frozen universal backbone + lightweight adapters":

Greener computation—no retraining trillion-parameter models from scratch
Lower barriers—small labs can innovate atop public frozen models
Democratized innovation—the interface layer becomes the new competitive focus

This doesn't end scaling but opens a hybrid era: core backbones grow moderately while peripheral tasks fall to efficient modularity—like a grand theater that stops building higher walls and instead masters lighting, curtains, and props to stage endless plays on one set.

Epilogue

The most striking lesson is not how many layers were stacked, but that the thinnest curtain and smallest memory can awaken such rich performance. ETH reveals eternal memory behind apparent forgetting; Apple's single attention layer bridges understanding and creation. Future AI may resemble a LEGO castle of frozen universal cortices and lightweight interfaces—efficient, green, and inclusive, as the "less is more" story just begins.

References

1. Asymptotic Analysis of Shallow and Deep Forgetting in Replay with Neural Collapse. arXiv:2512.07400, 2025. 2. Adapting Pretrained Visual Encoders for Image Generation. arXiv:2512.07829, 2025. 3. AI Giants Rethink Model Training Strategy as Scaling Laws Break Down. The Batch, DeepLearning.AI. 4. Can AI scaling continue through 2030? Epoch AI Blog. 5. The End of Scaling Laws: How Harvard's "Scaling Laws for Precision" Revolutionizes LLM Training. Towards AI, 2025.