[论文] Temporal Self-Distillation: Learning Visual State Tracking in Videos W...

论文概要

研究领域: CV
作者: Shravan Venkatraman, Wenshuai Zhao, Mohammad Hassan Vali
发布时间: 2026-09-03
arXiv: 2609.04203

中文摘要

我们提出了 S³T（时间上的自监督自蒸馏），这是首个完全自洽的连续视频状态跟踪框架。我们的方法将时间采样密度视为特权信息——基于一个核心假设：对同一段视频的更密集采样能更精确地恢复运行状态。这个密集视角作为教师模型，而一个共享权重的稀疏视角学生模型学习匹配其下一 token 分布。模型自主生成目标，因此训练无需标注、独立教师模型或奖励信号，且不增加推理开销。在 LLaVA-OneVision-2-8B 上，S³T 作为单一模型将 VSTAT 准确率提升 +1.74，模型融合后提升 +2.38，加上视觉编码器适配后提升 +2.70，而先前的自进化方法对状态跟踪几乎无改善。从未标注合成视频学到的能力可迁移到真实视频，在 VSTAT-YouTube 状态跟踪问题上提升 +7.95，在 MVBench 动作计数上提升 +4.50。

原文摘要

We introduce S\(^3\)T (Self-Supervised Self-Distillation over Time), which is the first fully self-contained framework for continuous video state tracking. Our method treats temporal sampling density as privileged information based on the hypothesis that a denser view of the same clip recovers the running state more accurately. This view serves as the teacher, while a sparse-view student with the same weights learns to match its next-token distribution. The model generates its own target, so training requires no labels, separate teacher, or reward signal, and adds no inference cost.

自动采集于 2026-09-05

#论文 #arXiv #CV #小凯