Summary
This post introduces DnA (Denoising Attention), a paper on arXiv (2606.27372) by Ron Campos, Subhajit Maity, and Xin Li in computer vision. The authors observe that the softmax activation in multihead attention (MHA), the de facto standard for attention-based visual perception models, can produce noisy attention patterns that dilute relevant features and degrade performance. DnA addresses this by using a positive query to identify image features belonging to the correct class and a negative query to identify closely associated but irrelevant image features. These interactions are then projected into two distinct subspaces, promoting subspace separation and improved discriminability of learned representations. The post includes the paper summary, original abstract, and a link to the arXiv page for readers interested in the full technical details.
Paper Overview
Research area: Computer Vision
Authors: Ron Campos, Subhajit Maity, Xin Li
Published: 2026-06-27
arXiv: 2606.27372
Summary
The softmax activation in multihead attention (MHA) is the de facto standard for attention-based models in visual perception tasks. However, standard softmax can produce noisy attention patterns that dilute relevant features and degrade performance.
This paper proposes Denoising Attention (DnA):
- A positive query identifies which image features belong to the correct class.
- A negative query identifies closely associated but irrelevant image features.
- DnA projects these interactions into two distinct subspaces, promoting subspace separation and improved discriminability.
The full paper abstract is truncated in the source post; readers can find the complete version on the
arXiv page.
---
*Auto-collected on 2026-06-27*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178208169