YouDub-webui is an open-source, local-first video dubbing tool that translates and re-voices YouTube and Bilibili videos end-to-end. Built as a single-process, serial FastAPI pipeline with nine stages—download (yt-dlp), vocal separation (Demucs htdemucs_ft), speech recognition with word-level timestamps (Whisper large-v3-turbo), sentence segmentation, LLM-based translation, speech synthesis with zero-shot voice cloning (VoxCPM2), time-stretching alignment, mixing, and FFmpeg composition—it has reportedly processed over 20,000 videos. Translation uses a two-phase strategy: full-transcript preprocessing to build summaries, hotword glossaries and corrections, followed by concurrent sentence-level translation with strict style rules. Every stage caches its output to disk, enabling resume-on-failure. Unlike SaaS alternatives such as HeyGen or Rask.ai, YouDub keeps all data local, charges only OpenAI API costs for translation, and runs on a single consumer GPU (8GB+ VRAM recommended). Limitations include single-threaded processing, uniform voice cloning across speakers, and hard-burned subtitles.
YouDub-webui: A Nine-Stage Local Pipeline for AI Video Dubbing with Whisper, Demucs and VoxCPM2
Summary
YouDub-webui is an open-source, local-first video dubbing tool that translates and re-voices YouTube and Bilibili videos end-to-end. Built as a single-process, serial FastAPI pipeline with nine stages—download (yt-dlp), vocal separation (Demucs htdemucs_ft), speech recognition with word-level timestamps (Whisper large-v3-turbo), sentence segmentation, LLM-based translation, speech synthesis with zero-shot voice cloning (VoxCPM2), time-stretching alignment, mixing, and FFmpeg composition—it has reportedly processed over 20,000 videos. Translation uses a two-phase strategy: full-transcript preprocessing to build summaries, hotword glossaries and corrections, followed by concurrent sentence-level translation with strict style rules. Every stage caches its output to disk, enabling resume-on-failure. Unlike SaaS alternatives such as HeyGen or Rask.ai, YouDub keeps all data local, charges only OpenAI API costs for translation, and runs on a single consumer GPU (8GB+ VRAM recommended). Limitations include single-threaded processing, uniform voice cloning across speakers, and hard-burned subtitles.
This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177981458