SAM2Matting: Generalized Image and Video Matting Using SAM2 Tracking

Forum topic · 小凯 · 2026-06-27

Summary

SAM2Matting is a tracker-to-matting framework for generalized image and video matting, presented in arXiv paper 2606.27339 by Ruiqi Shen, Guangquan Jie, and Chang Liu. The authors observe that video matting remains difficult because of an inherent gap between high-level tracking, which requires frame-wise understanding, and low-level matting, which must capture extremely fine-grained details such as hair and semi-transparent edges. Existing approaches rely on expensive, narrowly-scoped video matting datasets, which limits out-of-domain generalization and weakens tracking robustness. SAM2Matting instead reuses the strong tracking capability of SAM2 (Segment Anything Model 2) to drive a matting pipeline, producing high-quality matting results without requiring costly dedicated video matting datasets. By decoupling tracking from matting, the framework improves robustness and generalization across domains for both image and video matting tasks.

Overview

Research area: Computer Vision (CV)
Authors: Ruiqi Shen, Guangquan Jie, Chang Liu
Published: 2026-06-27
arXiv: 2606.27339

Abstract (translated from the Chinese summary)

Despite impressive advances in image matting, video matting remains challenging due to the inherent gap between high-level tracking, which requires frame-wise understanding, and low-level matting, which focuses on extremely fine-grained details.

The authors propose SAM2Matting, a tracker-to-matting framework that leverages SAM2's tracking capability to generate high-quality matting results without requiring expensive video matting datasets.

Original Abstract (excerpt)

> Despite impressive advances in image matting, video matting remains challenging due to the inherent gap between high-level tracking, which requires frame-wise understanding, and low-level matting, which focuses on extremely fine-grained details. Existing methods attempt this with expensive and narrowly-scoped video matting datasets, which may limit out-of-domain generalization and compromise tracking robustness. We rethink the paradigm with SAM2Matting, a tracker-to-matting framework that advanc...

Key Points

Video matting is harder than image matting because it must simultaneously handle high-level tracking (frame-wise understanding) and low-level matting (fine-grained detail extraction).
Prior methods depend on expensive, narrowly-scoped video matting datasets, which can hurt out-of-domain generalization and tracking robustness.
SAM2Matting reframes the problem as tracker-to-matting: it reuses SAM2's strong tracking ability to drive matting, avoiding the need for costly video matting datasets.

*Source: arXiv 2606.27339*