Summary
This paper addresses sentence-level sign language translation (SLT) with a focus on real-time deployment rather than novel architectures. The author fine-tunes a SHuBERT-ByT5 translation stack on a 9,872-example subset of How2Sign, achieving a validation BLEU of 16.7, test BLEU of 15.9, and BLEURT of 44.7. The main contribution is a hardware-aware streaming system: a Raspberry Pi 4B reference client handles camera capture, on-device text display, and speech output, while compute-intensive perception and translation run on a CPU/GPU backend. Techniques including chunked ingestion, bounded queues, parallel perception, temporal re-ranking, and a sentence-boundary state machine reduce average post-finalization latency from 1.873 seconds to 1.354 seconds, a 27.71% improvement. The work demonstrates a practical pipeline for low-latency sign language translation on consumer hardware.
Paper Overview
- Field: NLP
- Author: Thanh-Hoang Nguyen Doan
- Published: 2026-07-10
- arXiv: 2607.09611
Summary
Most sign language understanding systems operate at the level of isolated gestures, which limits their usefulness in natural communication. This paper studies sentence-level sign language translation (SLT), with the primary goal of real-time deployment rather than proposing a new architecture.
The author fine-tunes a SHuBERT-ByT5 translation stack on a 9,872-example subset of How2Sign, reaching:
- Validation BLEU: 16.7
- Test BLEU: 15.9
- Test BLEURT: 44.7
The main contribution is a hardware-aware streaming system. A Raspberry Pi 4B reference client handles camera capture, local text display, and speech output, while compute-intensive perception and translation run on a CPU/GPU backend.
Key system techniques include:
- Chunked ingestion
- Bounded queues
- Parallel perception
- Temporal re-ranking
- A sentence-boundary state machine
Together, these reduce average post-finalization latency from
1.873 seconds to 1.354 seconds, a
27.71% improvement.
---
*Auto-collected on 2026-07-14.*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178395122