DnA: Denoising Attention for Visual Tasks

Forum topic · 小凯 · 2026-06-27

Summary

This post introduces DnA (Denoising Attention), a paper on computer vision by Ron Campos, Subhajit Maity, and Xin Li (arXiv:2606.27372). Standard multihead attention (MHA) relies on softmax activation, which can produce noisy attention patterns that dilute relevant features and degrade performance in visual perception tasks. DnA addresses this by introducing a positive query that identifies image features belonging to the correct class and a negative query that identifies closely associated but irrelevant image features. These interactions are then projected into two distinct subspaces, promoting subspace separation and improved discriminability. The result is cleaner attention patterns and better feature discrimination for vision models. The forum post includes the paper's abstract and links to the arXiv page for full details.

Paper Overview

Field: Computer Vision (CV) Authors: Ron Campos, Subhajit Maity, Xin Li Published: 2026-06-27 arXiv: 2606.27372

Abstract

The softmax activation in multihead attention (MHA) is the de facto standard for attention-based models in visual perception tasks. However, standard softmax can produce noisy attention patterns that dilute relevant features and degrade its performance.

In this paper, the authors propose Denoising Attention (DnA). First, a positive query identifies which image features belong to the correct class, while a negative query identifies closely associated but irrelevant image features. DnA then projects these interactions into two distinct subspaces, promoting subspace separation and improved discriminability.

Key Points

Softmax-based MHA can yield noisy attention patterns that dilute relevant visual features.
DnA introduces positive and negative queries to distinguish class-relevant features from closely associated but irrelevant ones.
Projecting interactions into separate subspaces enhances subspace separation and feature discriminability.
Full details available on arXiv:2606.27372.

---

*Auto-collected on 2026-06-27.*