Paper Overview
- Field: Machine Learning
- Author: Zhangyong Liang
- Published: 2025-04-29
- arXiv: 2504.20638
- Explicitly encode a hidden representation of the varying small parameters, ensuring that the corresponding solving tasks are serialized for simultaneous training.
- Segment the prediction results to construct task losses and introduce a novel gradient alignment metric to mitigate gradient conflicts.
- Guarantee that the dot product between the final update and each loss-specific gradient remains positive.
- Provide mathematical proof that the HRGrad method converges.
Summary
This paper proposes HRGrad, a harmonized rotational gradient method for simultaneously tackling multiscale time-dependent kinetic problems with varying small parameters. These parameters exhibit asymptotic transitions from microscopic to macroscopic physics, making it a challenging multi-task problem to solve over all ranges simultaneously. Solving tasks in different asymptotic regions often encounter gradient conflicts, which can lead to the failure of multi-task learning.
Key Techniques
Abstract (Original)
> In this paper, we propose a harmonized rotational gradient method, termed HRGrad, for simultaneously tackling multiscale time-dependent kinetic problems with varying small parameters. These parameters exhibit asymptotic transitions from microscopic to macroscopic physics, making it a challenging multi-task problem to solve over all ranges simultaneously. Solving tasks in different asymptotic regions often encounter gradient conflicts, which can lead to the failure of multi-task learning. To address this challenge, we explicitly encode a hidden representation of these parameters, ensuring that the corresponding solving tasks are serialized for simultaneous training. Furthermore, to mitigate gradient conflicts, we segment the prediction results to construct task losses and introduce a novel ...
*Auto-collected on 2026-04-29.*