Painting in the Fog: Can We See Through DiffusionGemma's Thoughts?
> "If you want to understand something complex, simplify it until you can understand it; then complexify it back." > — Richard Feynman
🌫️ Introduction: A Question of Transparency
Imagine standing in a huge gallery filled with paintings. They are breathtaking — stars swirling across canvases, a girl's eyes holding entire universes, cities melting into golden rivers at dusk. You admire them, then turn to the painter and ask: "How did you do it?"
The painter stays silent. He did paint these masterpieces, but he won't tell you his process. You secretly observe him working: he doesn't paint directly on the displayed canvas; he paints in a completely dark room first, then moves the finished work to the gallery. You never see the moment his brush actually moves. You know the result, but not the process.
That is our relationship with large AI models today.
When you ask ChatGPT to write a poem, Midjourney to paint a picture, or Sora to generate a video, you see the result — stunning, fluent, seemingly soulful. But you don't know what this "machine brain" actually went through while generating it. What was it "thinking"? What reasoning chain did it follow? Why did it choose this word and not that one?
This "invisible process" is the core of the reasoning transparency problem. It is not an abstract philosophical issue — it determines whether we can truly understand, trust, and control these increasingly powerful systems.
🧠 What Is Reasoning Transparency?
Imagine solving a math problem. Your scratch paper is full of steps: write down knowns, draw auxiliary lines, derive a first conclusion, substitute into a second equation... and finally reach the answer. If someone asks "how did you solve it," you hand over the scratch paper — every step visible.
That is the essence of reasoning transparency: whether the "scratch paper" of a system's intermediate thinking is visible to us, and whether we can read it.
Researchers divide this into two levels:
🪞 Variable Transparency
This is the question of whether the words on the scratch paper can be seen clearly.
For traditional autoregressive language models (like GPT-4), this is relatively easy. They write a story one token at a time: write the first token, pause; based on it, write the second, pause again. At every step you can see its "state" — what has been written — and inspect its internal neuron activations to guess why it chose the next word.
But for diffusion models (like Stable Diffusion, DALL-E), the situation is completely different. They don't write word by word; they repeatedly revise inside a vast abstract latent space. Like someone painting a huge canvas with eyes closed, adjusting the entire picture's colors and shapes with every stroke. Take a snapshot of what it's "thinking now"? It's just a pile of numbers — vectors floating in a continuous space of millions of dimensions. You have no idea what these numbers mean.
🧩 Algorithmic Transparency
This is the question of whether, even if you can read the scratch paper, you can understand the reasoning.
For example: in an autoregressive model writing poetry, you observe that when it writes the word "moon," a particular neuron activates. You hypothesize this neuron relates to "romance" or "night." That is algorithmic transparency — inferring the reasoning process from intermediate states.
For diffusion models this is much harder. Because at each step, the model may change every part of the whole image. What it "thinks" at step 10 may be a completely different concept from step 11. It does not reason "along a straight line"; it dances a complex dance in a high-dimensional space.
🎨 Diffusion Models: Painters in Thick Fog
The Magic from Noise to Image
The core idea of diffusion models is surprisingly elegant — even counterintuitive:
Take a perfect photo. Gradually add noise until it becomes pure TV static — nothing visible. Like dropping ink into clear water until the whole glass turns black.
Training a diffusion model means teaching it the reverse process: given a fully noised image, how to remove noise step by step and recover the original. Or more practically: given random noise, how to denoise over many steps into a brand-new image.
Concretely, the workflow is:
1. Input: a completely random noise vector (like TV static). 2. Denoising step 1: the model guesses "if I remove a little noise, what should this look like?" and outputs a slightly less noisy image. 3. Denoising step 2: it looks at the new state and continues denoising. 4. ... 5. Denoising step N: after dozens or hundreds of steps, the model outputs a clear image.
At each step, the model predicts: given the current "intermediate state" (neither pure noise nor a clear image), which direction should the "true" image lie?
The Maze of Continuous Latent Space
A key detail: diffusion models don't operate directly on pixels. They first compress images into a latent space — a lower-dimensional but still enormous continuous vector space.
Think of it as a giant invisible library, where each "book" represents a possible image. The denoising process is walking through this library from a "random shelf" step by step toward the "target book."
The problem: the books have no text labels. You don't know what image a location corresponds to; nearby vectors may represent completely different images. The space is continuous, high-dimensional, extremely complex — human intuition fails here entirely.
This is why the paper says DiffusionGemma "performs a larger fraction of its computation in continuous latent space."
Autoregressive vs Diffusion: Two Modes of Thinking
| Dimension | Autoregressive (GPT) | Diffusion | |------|-------------|-------------------| | Generation | Left to right, one token at a time | From noise to clarity, iterative denoising | | Space | Discrete token space | Continuous latent space | | Intermediate states | Explicit, readable text sequences | Fuzzy, unreadable numeric vectors | | Reasoning | Linear, one-directional | Global, multi-step iterative | | Transparency | Relatively easy to understand | Extremely hard |
An autoregressive model is like a meticulous writer: finish paragraph one, then write paragraph two based on it. You can check his work at every pause and even predict his next move.
A diffusion model is like someone painting in a dream: eyes closed, smearing the canvas repeatedly, adjusting the whole picture with each stroke. Ask "what are you thinking right now?" — he might answer "I'm adjusting the color of the 37th pixel in the upper-left corner," which doesn't help you understand his creative process at all.
📊 A Ruler for Opacity: Opaque Serial Depth
The paper proposes a clever metric: Opaque Serial Depth, quantifying how "opaque" a model's reasoning is.
The concept is simple: it measures how much serial computation happens between two interpretable states.
Metaphor: reading a detective novel. At the end of each chapter, the detective summarizes the clues so far, and you understand these summaries. If every chapter ends with a clear state summary, the book's "opaque serial depth" is low — you can always follow what the detective is doing.
But if the novel has only a beginning and an end, with hundreds of pages of ciphertext in between you can't read at all, its opaque serial depth is very high.
For an autoregressive model (Gemma 4), after each generated token you can inspect the model's state. Only one step of computation separates interpretable states. Low opaque serial depth.
For raw diffusion models, tens of denoising steps may separate interpretable states, each conducted in continuous latent space, outputting unreadable vectors. So the opaque serial depth is very high — the paper reports it is 28.6x that of Gemma 4.
This means: to understand DiffusionGemma's reasoning during image generation, you must cross a "dark zone" 28.6x longer than a language model's — where no understandable intermediate state is visible.
🔍 Finding One: Illuminating a Tunnel Through the Latent Space
How did the researchers address this?
One core method: mapping information flow between denoising steps through an interpretable token bottleneck.
The Intuition of the Token Bottleneck
Imagine at a noisy party, trying to hear a friend. You create a "bottleneck": your friend shouts, everyone else is quiet. Only your friend's voice passes through.
In diffusion models, the "token bottleneck" does something similar: after each denoising step, researchers force the model through a bottleneck that retains only a small set of interpretable, discrete "concept tokens" (similar to text tokens). These concept tokens are the abstract concepts the model is "really thinking" at that step.
Concretely, the researchers found: although diffusion models work in continuous latent space, the information flow of each denoising step can be compressed onto a set of interpretable tokens. Like placing a filter in a great river — what remains on the filter is the core information.
Striking Results
Opaque serial depth drops from 28.6x to 1.1x — nearly matching autoregressive Gemma 4!
This means: although diffusion models work in continuous latent space, the information they truly need to pass between denoising steps can be compressed into interpretable units similar to language model tokens. The diffusion model's "surface" opacity largely comes from representation (continuous vectors vs discrete tokens), not from fundamental incomprehensibility.
This is a profound finding: it suggests diffusion models' reasoning may not be as mysterious as we imagined.
🧩 Finding Two: The Algorithmic Transparency Gap Is Harder to Cross
Improving variable transparency is only the first step. Algorithmic transparency — understanding how the model uses intermediate states to make decisions — remains a bigger challenge.
The paper argues diffusion models are inherently harder than autoregressive ones here. The reason is deep:
The Power of Global Modification
In autoregressive models, once a token is generated, it usually isn't changed (at least in standard greedy decoding). The model "looks forward": it decides the next step based on what's already generated. This unidirectionality makes reasoning relatively easy to trace — like a chain you can follow from one end to the other.
In diffusion models, each denoising step may change every part of the image. At step 10 the model decides "this should be sky"; at step 11 it may reconsider and turn the whole sky into ocean. This "global modification" power gives diffusion models great flexibility — but makes the reasoning process extremely complex.
The paper describes how diffusion models can "implement complex distributed algorithms during denoising." It may simultaneously perform different "reasoning" in different image regions, with these reasonings interrelated. Like a symphony orchestra: each instrument plays its own melody, but together they form a complete piece — you can't understand the whole by listening to one instrument.
Three Striking Discoveries
To explore this "black box," the researchers ran a series of case studies, finding three diffusion-specific phenomena:
#### 🔄 1. Non-Chronological Reasoning
In autoregressive models, reasoning is usually "chronological": first paragraph, then second, logic flowing left to right. Diffusion models don't follow this order. Researchers found they may decide global, high-level features early in denoising ("this is a landscape," "the subject is centered"), then fill in details later ("how many leaves this tree has").
It's like a painter first sketching a blurry outline, then filling in details. But the key difference: a diffusion model may "know" where certain details belong even before the outline is clear. It doesn't "build" an image step by step — it considers multiple aspects simultaneously, more globally and in parallel.
#### 🌫️ 2. Token and Sequence Smearing
Concept representations in diffusion models are often not "clean one-to-one." A specific concept (like "red") may not be encoded in a single token, but "smeared" across a combination of multiple tokens. Moreover, the smearing may span multiple denoising steps — concepts "flow" and "transform" between steps.
Imagine watercolor bleeding on wet paper: a drop of pigment doesn't keep sharp boundaries; it spreads and mixes with other colors. In the latent space of diffusion models, concept representations behave the same — not discrete and definite, but continuous and interpenetrating.
This makes "tracing" a concept's evolution through reasoning extremely hard. You can't simply say "at step 5 the model thought red, at step 10 it became orange." The color may be a gradient, existing in multiple states across many steps simultaneously.
#### 🌉 3. Intermediate-Context Reasoning
This is among the most surprising findings. Diffusion models exploit context information that exists only in intermediate steps — information that completely disappears from the final output.
Imagine writing a novel. Your draft includes a character betraying the protagonist. Then you revise, deleting the betrayal, ending the story harmoniously. In the final version, the betrayal never happened. Yet that initially conceived betrayal may have influenced certain dialogues you wrote afterward — even dialogues that superficially have nothing to do with betrayal.
Diffusion models do the same. At some denoising step, the model may temporarily "believe" the image should contain a feature (like a shadow), then change its mind in later steps and delete it. But that "deleted" idea may already have influenced other parts of the generation — those effects persist in the final output, while their "source" has vanished.
This means: even if you can see every intermediate state, you may not fully understand the final output, because some reasoning happened in "already-vanished" context.
🛡️ Finding Three: Monitorability — Transparency's Ultimate Purpose
Why care about transparency? Theoretical curiosity is part of it, but what matters more is practical application. The paper tested a key application: monitorability.
What Monitorability Means
Monitorability asks: given the model's internal states (intermediate reasoning), can we use them for downstream tasks — like detecting whether the model is telling the truth, contains bias, or has been manipulated?
Concrete example: a model generates an image. You want to know whether it contains hidden bias (like always drawing doctors as male). If you can monitor the reasoning process, you might detect early in generation that when the model considers the concept "doctor," its association with "male" is stronger.
Or in safety scenarios: detecting whether a model has been "jailbroken" — whether someone is trying to make it generate harmful content. With visibility into the reasoning process, you might spot anomalous internal activation patterns.
Encouraging Results
The paper tested DiffusionGemma's monitorability and compared it with Gemma 4. The result: they are very similar.
This means: despite working in latent space, despite more complex reasoning, their internal states remain equally useful for downstream monitoring tasks.
This is a pragmatic, encouraging finding: even without fully understanding diffusion models' reasoning, we can still effectively use these processes to supervise and control the models.
🎯 Core Conclusion: Diffusion Transparency Is Better Than Assumed
The paper's core message:
> Diffusion models' reasoning transparency is misleadingly underestimated by their representation (continuous latent space). Once we re-represent these processes properly (e.g., via a token bottleneck), their comprehensibility approaches that of autoregressive models.
Specifically, the paper's three core contributions:
1. Variable transparency can be greatly improved: via an interpretable token bottleneck, opaque serial depth drops from 28.6x to 1.1x.
2. Algorithmic transparency remains challenging: diffusion models' global modification ability, non-chronological reasoning, concept smearing, and intermediate-context reasoning make their reasoning harder to understand than autoregressive models'. But through case studies, we're beginning to glimpse the "tip of the iceberg."
3. Monitorability is not harmed: even with more complex reasoning, diffusion models' intermediate states are equally effective for downstream supervision.
🌌 Deeper Reflection: Transparency Is a Capability — and a Responsibility
Why Does Transparency Matter?
Imagine an employee who is brilliant and highly efficient, but never explains his work process. Every report he submits is flawless, but you don't know how he wrote it. Did he plagiarize? Hide errors? Cut corners?
Would you trust this employee? Let him handle sensitive information? Entrust him with the company's future?
AI models are this "employee." When they help with medical diagnosis, legal decisions, financial analysis, content moderation, we need to know how they think. Not out of voyeurism, but because trust requires understanding.
Richard Feynman said: "What I cannot create, I do not understand." For AI, we can reverse it: if we cannot understand how it creates, we have not truly mastered it.
Transparency Is Not the End — It Is the Beginning
This paper is a starting point, not a conclusion. It tells us the diffusion model "black box" is not impenetrable. With clever methods (like the token bottleneck), we can illuminate most of the space inside.
But many questions remain:
- Does the token bottleneck lose important information?
- What is the specific mechanism of non-chronological reasoning?
- How does intermediate-context reasoning affect the quality and safety of final outputs?
- How do these findings apply to real-world model monitoring and alignment?
Implications for the Future of Diffusion Models
Safer AI systems: if we can monitor diffusion models' reasoning, we can detect the intent to generate harmful content earlier — intervening before content is produced.
More explainable AI art: when AI paints, we might ask "why did you choose this color?" and get a meaningful answer — "because at the third denoising step I recognized a sunset scene, and sunset scenes usually contain warm tones."
Better model design: understanding diffusion models' reasoning can help design better architectures. For instance, if we know diffusion models tend to use global context at intermediate steps, we might design specialized mechanisms to exploit this.
📚 References
1. Engels, J., McDougall, C., Chughtai, B., et al. "How Transparent is DiffusionGemma?" arXiv:2606.20560, 2026. 2. Chen, T., et al. "Diffusion Models: A Comprehensive Survey of Methods and Applications." ACM Computing Surveys, 2024. 3. Meng, K., et al. "Locating and Editing Factual Associations in GPT." NeurIPS, 2022. 4. Nanda, N., et al. "A Circuit Tracing Toolkit for Mechanistic Interpretability." Distill, 2023. 5. Ho, J., et al. "Denoising Diffusion Probabilistic Models." NeurIPS, 2020. 6. Rombach, R., et al. "High-Resolution Image Synthesis with Latent Diffusion Models." CVPR, 2022.
*Daily Paper Recommendation | 2026-06-21*