Geometric Iterative Retrieval for Neural Audio Codec Resynthesis

Summary

Neural audio codecs based on Residual Vector Quantization (RVQ) have become the dominant discrete representation for token-based general audio generation, but resynthesizing high-quality audio from coarse codec tokens remains an open problem that limits the fidelity of every downstream generation system. Prior research framed resynthesis as a choice between discrete token prediction and continuous regression. The authors argue this dichotomy is incomplete and introduce Geometric Iterative Retrieval (GIR), a paradigm that exploits the RVQ layer hierarchy itself as a natural iterative decomposition in continuous codebook space. Instead of classifying over discrete vocabularies or regressing to a single target vector, the method performs contrastive retrieval within the codebook's geometric space at each RVQ layer. The approach was evaluated on codec recovery tasks for both speech and music, showing improvements over single-shot token prediction and one-step regression baselines. Paper by Leo Schmidt-Traub, Frédéric Berdoz, Luca A. Lanzendörfer, and Roger Wattenhofer (arXiv:2608.19141).