[论文] HRIL: Learning Multimodal Synergy via Higher-Order Tensor Modeling

论文概要

研究领域: ML
作者: Qun Dai, Liangjian Wen, Jiang Duan, Yong Dai, Dongkai Wang, Maolin Wang, Mingjie Wang, Jianzhuang Liu, He Yan, Zhao Kang
发布时间: 2026-10-08
arXiv: 2610.12393

中文摘要

自监督多模态表示学习已在多个领域取得巨大成功，但捕捉跨模态的协同信息（synergy）仍然困难——协同信息与各模态共享的信息不同，任务相关信号仅在多模态的联合配置下才涌现，无法从任何单一模态中恢复。本工作的关键观察是：协同信息体现在模态间的高阶统计依赖中，这为显式建模联合交互提供了有原则的目标。受此启发，我们提出 HRIL（高阶表示与信息学习）：在模态嵌入上构建经验交叉矩张量来表示多方交互，采用 Tucker 分解获得核心张量，并配合协同感知正则化器防止能量集中、保持高阶耦合容量。在受控协同任务和真实基准上的实验表明，HRIL 一致优于现有对比式多模态方法，在由协同交互主导的任务上提升尤为显著。

原文摘要

Self-supervised multimodal representation learning has achieved remarkable success across diverse domains, yet capturing synergistic information remains challenging due to the complexity of cross-modal interactions. Unlike the shared information across individual modalities, synergy arises when task-relevant signals emerge only from the joint configuration of multiple modalities and cannot be recovered from any modality in isolation. This work focuses on how to preserve the information capacity for such synergistic signals in multimodal representations. The key observation is that synergistic information is reflected in higher-order statistical dependence among modalities, which provides a principled target for explicitly modeling joint interactions. Motivated by this insight, we propose H...

自动采集于 2026-10-12

#论文 #arXiv #ML #小凯