[论文] [论文] MedVTok: A General-Purpose Medical Visual Tokenizer

论文概要

研究领域: CV
作者: Chenglong Ma, Yuanfeng Ji, Junzhi Ning, Jiyao Liu, Yujie Wei et al.
发布时间: 2026-10-26
arXiv: 2610.35842

中文摘要

多模态医疗 AI 需要通用视觉接口，把异构影像证据编码为可建模患者状态的表征。但现有医疗视觉 tokenizer 往往绑定特定维度与影像模态，迫使多模态系统依赖碎片化、难复用的表征。我们提出 MedVTok——通用视觉 tokenizer，把多种影像模态的 2D 图像与 3D 体数据映射到统一 token 空间。构建它有两个关键挑战：跨维度保持解剖结构、跨模态捕捉临床语义。为此引入 (i) 切片差分正则化，显式建模以往逐像素/逐体素监督缺失的切片间解剖连贯性；(ii) 多专家表征对齐，融合多个医疗专家编码器的知识同时保留各模态特有的互补信息。在 9 种影像模态的 2800 万张图像与 7 万个体数据上训练，MedVTok 以单一 tokenizer 支持分类、检索、分割、合成、视觉问答与报告生成，跨 24 个基准取得 SOTA，证明了统一医疗图像维度、模态与任务用法的可扩展 token 接口。

原文摘要

Multimodal medical AI requires a common visual interface for encoding heterogeneous imaging evidence for patient condition modeling. However, existing medical visual tokenizers are often tied to a specific dimension and imaging modality, forcing multimodal systems to rely on fragmented and poorly reusable representations. We introduce MedVTok, a general-purpose visual tokenizer that maps 2D images and 3D volumes from diverse imaging modalities into a unified token space. Building such a tokenizer poses two key challenges: preserving anatomical structure across dimensions and capturing clinical semantics across modalities. To this end, MedVTok introduces (i) slice differential regularization, which explicitly models inter-slice anatomical coherence missing in previous per-pixel/per-voxel supervision, and (ii) multi-expert representation alignment, which integrates knowledge from multiple medical expert encoders while retaining modality-specific complementary knowledge. Trained on 28M images and 70K volumes across 9 imaging modalities, MedVTok supports classification, retrieval, segmentation, synthesis, visual question answering, and medical report generation through a single tokenizer. Across 24 benchmarks, MedVTok achieves state-of-the-art performance, demonstrating a scalable token interface that unifies medical image dimension, modality, and task usage.

自动采集于 2026-10-28

#论文 #arXiv #CV #小凯