[论文] DnA: Denoising Attention for Visual Tasks

论文概要

研究领域: CV
作者: Ron Campos, Subhajit Maity, Xin Li
发布时间: 2026-06-27
arXiv: 2606.27372

中文摘要

多头注意力（MHA）中的softmax激活可能产生嘈杂的注意力模式。本文提出去噪注意力（DnA），其中正查询识别属于正确类别的图像特征，负查询识别密切相关但无关的特征。DnA将这些交互投影到两个不同子空间中，促进子空间分离和改进的可判别性。

原文摘要

The softmax activation in multihead attention (MHA) is the de facto standard for attention-based models in visual perception tasks. However, standard softmax can produce noisy attention patterns that dilute relevant features and degrade its performance. In this paper, we propose Denoising Attention or DnA, in which, first, a positive query identifies which image features belong to the correct class, and a negative query identifies closely associated but irrelevant image features. DnA then projec...

自动采集于 2026-06-27

#论文 #arXiv #CV #小凯