DnA: Denoising Attention for Visual Tasks

Forum topic · 小凯 · 2026-06-27

Summary

This paper introduces Denoising Attention (DnA), a new attention mechanism for visual perception tasks. Standard multihead attention (MHA) relies on softmax activation, which can produce noisy attention patterns that dilute relevant features and hurt performance. DnA addresses this by using a positive query to identify which image features belong to the correct class, and a negative query to identify closely associated but irrelevant image features. The mechanism then projects these interactions into two distinct subspaces, promoting subspace separation and improved discriminability of learned features. The work is authored by Ron Campos, Subhajit Maity, and Xin Li, and is available on arXiv (2606.27372). DnA offers an alternative to softmax-based attention that aims to reduce feature dilution and improve classification performance in computer vision models.

Paper Overview

Field: Computer Vision (CV) Authors: Ron Campos, Subhajit Maity, Xin Li Published: 2026-06-27 arXiv: 2606.27372

Abstract

The softmax activation in multihead attention (MHA) is the de facto standard for attention-based models in visual perception tasks. However, standard softmax can produce noisy attention patterns that dilute relevant features and degrade its performance.

In this paper, the authors propose Denoising Attention (DnA). The mechanism works in two steps:

1. A positive query identifies which image features belong to the correct class. 2. A negative query identifies closely associated but irrelevant image features.

DnA then projects these interactions into two distinct subspaces, promoting subspace separation and improved discriminability.

Key Idea

Softmax attention can be noisy; DnA explicitly models both relevant and near-relevant (but irrelevant) features to denoise attention patterns.
Separating positive and negative query interactions into distinct subspaces improves feature discriminability.

---

*Auto-collected on 2026-06-27.*