REDDIT: Correcting Timestamp Drift in Autoregressive ASR via Replay-Based Distribution Editing

Forum topic · 小凯 · 2026-07-08

Summary

Modern autoregressive ASR systems can emit timestamps as decoded tokens, enabling timestamped transcription without frame-level aligners or post-processing. This paper identifies a key failure mode: generated timestamps drift during long non-speech spans, where the transcript may remain plausible but the decoded timeline diverges from the audio. The authors propose REDDIT (Replay-based Distribution Editing), a lightweight two-stage post-training framework that corrects timestamps while avoiding catastrophic forgetting. First, it edits timestamp targets under the model's own replayed decoder context while matching a frozen base distribution for non-timestamp tokens; then it applies a short editing-prefix refinement stage. On Whisper-tiny, using only 34.9 hours of target correction audio and updating just 1.6% of model parameters, REDDIT raises long-gap mIoU from 38.7% to 95.0%, reduces mixed-gap out-of-domain AAS from 2752ms to 223ms, and preserves CV-en MER at 41.3%—compared to 524.2% under naive SFT decoder tuning. Paper: arXiv 2607.05364, by Cheng-Kang Chou, Ming-To Chuang, Ke-Han Lu, Chan-Jan Hsu, and Hung-yi Lee (July 2026).

Paper Overview

Field: NLP (speech recognition)
Authors: Cheng-Kang Chou, Ming-To Chuang, Ke-Han Lu, Chan-Jan Hsu, Hung-yi Lee
Published: 2026-07-06
arXiv: 2607.05364

Background

Modern autoregressive ASR systems can emit timestamps as decoded tokens, enabling timestamped transcription without a frame-level aligner or inference-time post-processing. However, this paper observes that these generated timestamps drift across long non-speech spans: the transcript text may still look reasonable, but the decoded timeline diverges from the actual audio.

Method: REDDIT

REDDIT (Replay-based distribution EDDITing) is a lightweight two-stage post-training framework designed to correct timestamps while avoiding catastrophic forgetting:

1. Stage 1 – Timestamp editing: Edit timestamp targets under the model's own replayed decoder context, while matching a frozen base distribution for non-timestamp tokens. 2. Stage 2 – Prefix refinement: Apply a short editing-prefix refinement stage.

Results (Whisper-tiny)

Using only 34.9 hours of target correction audio and updating just 1.6% of model parameters:

Long-gap mIoU: 38.7% → 95.0%
Mixed-gap out-of-domain AAS: 2752ms → 223ms
CV-en MER preserved at 41.3% (vs. 524.2% for naive SFT decoder tuning)

Takeaway

REDDIT shows that targeted, parameter-efficient distribution editing can fix timestamp drift in token-based ASR without sacrificing general transcription quality or triggering catastrophic forgetting.

---

*Auto-collected on 2026-07-06.*