Summary
DnA (Denoising Attention) is a paper by Ron Campos, Subhajit Maity, and Xin Li proposing a modification to multihead attention (MHA) for visual perception tasks. The authors argue that the standard softmax activation, the de facto standard in attention-based models, can produce noisy attention patterns that dilute relevant features and degrade performance. DnA addresses this by introducing a positive query that identifies image features belonging to the correct class, alongside a negative query that identifies closely associated but irrelevant features, enabling the model to denoise its attention patterns. The work is available on arXiv as 2606.27372 and was shared on zhichai.net on 2026-06-27.
Paper Overview
Research area: Computer Vision (CV)
Authors: Ron Campos, Subhajit Maity, Xin Li
Posted: 2026-06-27
arXiv:
2606.27372Abstract
The softmax activation in multihead attention (MHA) is the de facto standard for attention-based models in visual perception tasks. However, standard softmax can produce noisy attention patterns that dilute relevant features and degrade its performance. In this paper, the authors propose
Denoising Attention (DnA), in which, first, a positive query identifies which image features belong to the correct class, and a negative query identifies closely associated but irrelevant image features. DnA then projects...
*Note: The source excerpt is truncated; see the full paper on arXiv for complete details.*
---
*Auto-collected on 2026-06-27*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178208192