[论文] How Transparent is DiffusionGemma?

论文概要

研究领域: ML 作者: Joshua Engels, Callum McDougall, Bilal Chughtai 发布时间: 2025-06-20 arXiv: 2506.16807

中文摘要

大型语言模型推理透明度对理解模型决策、缓解误用与失调，以及调试意外行为至关重要。然DiffusionGemma在其连续潜在空间中执行大量计算——此是否使其推理更不透明？

本文将透明度分解为两部分：变量透明度（我们是否理解模型计算状态之中间快照）与算法透明度（我们能否利用这些快照重构模型得出输出之过程）。初看之下，DiffusionGemma之变量透明度欠佳：其不透明串行深度（可解释模型状态间之串行计算量）似乎比自回归Gemma 4模型高28.6倍。然作者示之，降噪步骤间流动之信息可经一可解释令牌瓶颈映射，而不降低下游性能。将这些中间状态视为可解释后，不透明串行深度降至仅为Gemma 4之1.1倍。

扩散模型之算法透明度较自回归模型更难，因为画布中所有令牌预测在每一步降噪时皆可改变，允许模型在降噪期间实现复杂分布式算法。作者开展一系列可解释性案例研究，发现扩散特有新现象之初步证据，如非时序推理、令牌与序列涂抹，以及中间语境推理。最后，彼等测试可监控性，发现DiffusionGemma与Gemma 4之可监控性相近。

原文摘要

LLM reasoning transparency is a critical affordance for understanding model decisions, mitigating misuse and misalignment, and debugging surprising model behaviors. However, DiffusionGemma performs a larger fraction of its computation in a continuous latent space; does this make its reasoning less transparent? We study this question by decomposing transparency into two components: variable transparency, whether we understand intermediate snapshots of a model's computational state; and algorithmic transparency, whether we can use these snapshots to reconstruct the process by which the model arrived at its outputs. Naively, DiffusionGemma has poor variable transparency: its opaque serial depth, the amount of serial computation that occurs in between interpretable model states, seems at first...

--- *自动采集于 2026-06-20*

#论文 #arXiv #ML #小凯