[论文] FlexUMM: Flexible Representation Learning for Unified Multimodal Mo...

论文概要

研究领域: ML
作者: Yinyi Luo, Haoyue Bai, Yi Ge, Nilton Cesar Rojas Vales, Marios Savvides, Jindong Wang
发布时间: 2026-10-19
arXiv: 2610.22682

中文摘要

统一多模态模型（UMM）能在单一骨干内整合理解与生成，但两大能力常产生表征冲突，核心问题是：表征与计算应如何在功能间共享？现有 UMM 多依赖基本固定的结构——不同输入与功能遵循同样的深度和参数共享模式。本文主张这一假设过于受限：更有效的统一应共享支撑跨功能通信的计算，而把会变成任务或输入特有的容量专门化。FlexUMM 动态决定何时保留共享计算、何时路由专门容量、何时停止进一步处理。证据显示这种灵活分配沿两个互补轴改善 UMM：通过减少功能间的有害竞争提升准确率，通过在输入表征足够成熟时避免不必要计算提升效率。除从预训练模型做简单的架构统一外，FlexUMM 还可作为模型无关的功能插件，让现有 UMM 更统一、更强、更高效。代码已开源。

原文摘要

Unified multimodal models (UMMs) can integrate understanding and generation within a single backbone. However, it is commonly known that these two capabilities can cause conflicting representations, leading to a central question: how representation and computation should be shared across functionalities? Existing UMMs often rely on a largely fixed structure, where different inputs and functionalities follow the same depth and parameter-sharing pattern. We argue that this assumption is too restrictive: more effective unification requires sharing the computation that supports cross-functional communication while specializing the capacity that becomes task- or input-specific. We introduce FlexUMM, a flexible UMM framework that dynamically decides when to preserve shared computation, when to route specialized capacity, and when to stop further processing. Evidence shows that this flexible allocation improves UMMs along two complementary axes. It improves accuracy by reducing harmful competition between functionalities, and it improves efficiency by avoiding unnecessary computation once an input representation is sufficiently mature. Beyond simple architectural unification from the pre-trained models, FlexUMM works as a model-agnostic, functional plug-in to make existing UMMs more unified, stronger, and efficient. Code is available at: this https URL.

自动采集于 2026-10-19

#论文 #arXiv #ML #小凯