ViTeX-Bench: Benchmarking High-Fidelity Video Scene Text Editing

Forum topic · 小凯 · 2026-10-02

Summary

ViTeX-Bench is a new benchmark suite for video scene text editing—replacing text on scene surfaces such as storefront signs, whiteboards, and product labels while preserving surrounding content, motion, and camera dynamics. The suite includes the ViTeX dataset of 387 real-world 720p videos with text-region masks and editing instructions: 230 videos with audited, pipeline-generated paired edits for training and 157 forming a frozen evaluation set. A three-axis evaluation protocol measures text correctness, visual and temporal quality, and edit locality via 13 metrics, each axis with one primary metric, plus Pareto-based trade-off analysis. OCR calibration, human evaluation, and annotation sensitivity analysis support score interpretation. Across eight baselines from four editing families, accurate text, temporal stability, and scene preservation remain hard to achieve simultaneously. The authors also release ViTeX-Edit-14B, an open-source reference editor fine-tuned on the paired training set with motion-aligned glyph-video conditioning, achieving 0.688 character accuracy—the highest mean among evaluated video-native editors—and the lowest comparable text-crop Warp score. Paper: arXiv 2609.26755.

Paper Overview

Field: Computer Vision Authors: Xinghao Chen, Xiangbo Gao, Jiongze Yu, et al. Published: 2026-09-30 arXiv: 2609.26755

Introduction

Recent video generation is increasingly realistic and controllable, yet video editing remains less developed—particularly for precise local edits that must preserve the original scene dynamics. Video scene text editing replaces text on scene surfaces, such as storefront signs, whiteboards, and product labels, while preserving the surrounding content, motion, and camera dynamics. Although scene text editing is well studied for images, video scene text editing that achieves high visual quality, temporal consistency, and edit locality remains underexplored. Existing resources offer limited paired real-video data, and general video-editing metrics do not directly measure whether the requested text remains correct over time.

ViTeX-Bench

ViTeX-Bench is a benchmark suite comprising:

ViTeX-Dataset: 387 real-world 720p videos with text-region masks and editing instructions. Of these, 230 provide audited, pipeline-generated paired edits for training, and 157 form a frozen evaluation set.
A three-axis evaluation protocol measuring text correctness, visual and temporal quality, and edit locality via 13 metrics, each axis anchored by one primary metric, with Pareto-based comparison to analyze trade-offs.
Validation: OCR calibration, human evaluation, and annotation sensitivity analysis support interpretation of the scores.

Baseline Findings

Across eight baseline methods from four editing families, accurate text, temporal stability, and scene preservation remain difficult to achieve simultaneously.

ViTEX-Edit-14B Reference Editor

The authors also release ViTEX-Edit-14B, an open-source reference editor fine-tuned on the paired training set using motion-aligned glyph-video conditioning. It achieves 0.688 character accuracy—the highest mean among evaluated video-native editors—and the lowest comparable text-crop Warp score among original editor outputs.

---

*Auto-collected on 2026-10-02*