Interference-Aware Multi-Task Unlearning: Gradient Projection Framework (arXiv 2505.01257)

Forum topic · 小凯 · 2026-05-21

Summary

A paper by Ying-Hua Huang, Rui Fang, and Hsi-Wen Chen (arXiv:2505.01257) introduces multi-task unlearning, extending machine unlearning beyond single-task settings to models trained on multiple tasks with shared backbones. The authors identify two settings: full-task unlearning, which removes a target instance from all tasks, and partial-task unlearning, which removes supervision only from selected tasks. They show that shared parameters couple the forget and retain sets, causing task-level interference on non-target tasks and instance-level interference on other instances. To address this, they propose an interference-aware framework combining task-aware gradient projection, which constrains updates within task-specific subspaces, with instance-level gradient orthogonalization, which reduces conflicts between forget and retain signals. Experiments on two multi-task computer vision benchmarks across five tasks demonstrate effective unlearning with strong generalization, reducing unwanted interference (UIS) by 30.3% in full-task unlearning and 52.9% in partial-task unlearning compared with the strongest baseline.

Paper Overview

Field: cs.AI
Authors: Ying-Hua Huang, Rui Fang, Hsi-Wen Chen
Published: 2026-05-21
arXiv: 2505.01257

Abstract

Machine unlearning aims to remove the contribution of designated training data from a trained model while preserving performance on the remaining data. Existing work mainly focuses on single-task settings, whereas modern models often operate in multi-task setups with shared backbones, where removing supervision for one task or instance can unintentionally affect others.

Key Contributions

Multi-task unlearning formulation with two settings:
Full-task unlearning: removes a target instance from all tasks.
Partial-task unlearning: removes supervision only from selected tasks.
Problem identification: shared parameters couple the forget and retain sets, causing task-level interference on non-target tasks and instance-level interference on other instances.
Interference-aware framework combining:
Task-aware gradient projection: constrains updates within task-specific subspaces.
Instance-level gradient orthogonalization: reduces conflicts between forget and retain signals.

Results

Experiments on two multi-task computer vision benchmarks across five tasks show the method achieves effective unlearning while maintaining strong generalization:

30.3% reduction in UIS compared with the strongest baseline in full-task unlearning.
52.9% reduction in UIS in partial-task unlearning.

---

*Auto-collected on 2026-05-21*