Paper Overview
Research Area: Machine Learning Authors: Qun Dai, Liangjian Wen, Jiang Duan, Yong Dai, Dongkai Wang, Maolin Wang, Mingjie Wang, Jianzhuang Liu, He Yan, Zhao Kang Published: 2026-10-08 arXiv: 2610.12393
Summary
Self-supervised multimodal representation learning has achieved remarkable success across diverse domains, yet capturing synergistic information remains challenging due to the complexity of cross-modal interactions. Unlike the shared information across individual modalities, synergy arises when task-relevant signals emerge only from the joint configuration of multiple modalities and cannot be recovered from any modality in isolation. This work focuses on how to preserve the information capacity for such synergistic signals in multimodal representations.
The key observation is that synergistic information is reflected in higher-order statistical dependence among modalities, which provides a principled target for explicitly modeling joint interactions.
Method: HRIL
Motivated by this insight, the authors propose HRIL (Higher-order Representation and Information Learning):
- Construct an empirical cross-moment tensor over modality embeddings to represent multi-way interactions
- Apply Tucker decomposition to obtain a core tensor capturing higher-order structure
- Add a synergy-aware regularizer that prevents energy concentration and preserves higher-order coupling capacity
Results
Experiments on controlled synergy tasks and real-world benchmarks show that HRIL consistently outperforms existing contrastive multimodal methods, with especially significant improvements on tasks dominated by synergistic interactions.
---
*Auto-collected on 2026-10-12*