Summary
This post introduces S³T (Self-Supervised Self-Distillation over Time), a fully self-contained framework for continuous video state tracking in computer vision, by Shravan Venkatraman, Wenshuai Zhao, and Mohammad Hassan Vali (arXiv:2609.04203). The method treats temporal sampling density as privileged information: a densely sampled view of the same video clip serves as a teacher, while a sparse-view student model with shared weights learns to match the teacher's next-token distribution. Because the model generates its own training targets, S³T requires no labels, no separate teacher model, and no reward signal, and adds no inference-time overhead. Applied to LLaVA-OneVision-2-8B, S³T improves VSTAT accuracy by +1.74 as a single model, +2.38 after model merging, and +2.70 with additional visual encoder adaptation, whereas prior self-evolution methods showed almost no improvement on state tracking. Capabilities learned from unlabeled synthetic videos transfer to real videos, yielding gains of +7.95 on VSTAT-YouTube state tracking questions and +4.50 on MVBench action counting.
Paper Overview
Research Area: Computer Vision (CV)
Authors: Shravan Venkatraman, Wenshuai Zhao, Mohammad Hassan Vali
Published: 2026-09-03
arXiv: 2609.04203
Introduction
The authors introduce S³T (Self-Supervised Self-Distillation over Time), the first fully self-contained framework for continuous video state tracking.
Method
- S³T treats temporal sampling density as privileged information, based on the hypothesis that a denser view of the same clip recovers the running state more accurately.
- The dense view serves as the teacher, while a sparse-view student with the same weights learns to match its next-token distribution.
- The model generates its own target, so training requires no labels, no separate teacher, and no reward signal, and adds no inference cost.
Results
On LLaVA-OneVision-2-8B, S³T improves VSTAT accuracy:
- +1.74 as a single model
- +2.38 after model merging
- +2.70 with additional visual encoder adaptation
Prior self-evolution methods showed almost no improvement on state tracking.
Transfer to Real Videos
Capabilities learned from unlabeled synthetic videos transfer to real videos:
- +7.95 on VSTAT-YouTube state tracking questions
- +4.50 on MVBench action counting
---
*Auto-collected on 2026-09-05.*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178634484