SignRR: Retrieve and Refine Real Motion for Sign Language Production
Field: Computer Vision Authors: Fidel Omar Tito Cruz, Angie Sanchez Marquina, Summy Farfan, Gissella Bejarano Published: 2026-08-28 arXiv: 2608.28568
Overview
Sign language production (SLP) aims to generate continuous signing motion from spoken language, often through gloss-to-pose generation. Prior work mainly follows two paradigms:
- Generative models synthesize motion from a learned prior or from noise, without reference to an observed signing instance, making rare hand configurations and signer-specific articulation difficult to preserve.
- Retrieval-based methods reuse real, well-articulated motion segments, but concatenating segments from different signers and co-articulation contexts can introduce rhythm and style inconsistencies across the full sequence, not only at segment boundaries.
SignRR: Retrieve, then Refine
These limitations motivate a complementary solution: use retrieval to provide realistic articulation, and use learned refinement to impose the global coherence that retrieval alone lacks. SignRR therefore adopts a retrieve-and-refine paradigm—starting from real retrieved motion and refining it into a globally coherent signing sequence, rather than generating motion from scratch.
The framework: 1. Initializes motion from a dictionary of real sign language segments. 2. Refines the full sequence using a part-aware residual VQ-VAE, where residual quantization preserves fine-grained hand articulation and temporal length differences are handled in latent space.
Results
Experiments on PHOENIX14T and CSL-Daily show that SignRR achieves state-of-the-art back-translation performance while maintaining competitive pose quality.
---
*Auto-collected on 2026-09-01*