Revisiting Input Time-frequency Representations in Multi-pitch Estimation for Vocal Ensembles

Forum topic · 小凯 · 2026-10-06

Summary

This arXiv paper (2610.03656) by Junyoung Koh and Hao-Wen Dong revisits the choice of input time-frequency representation for multi-pitch estimation in vocal ensembles. Multi-pitch estimation is difficult because singers occupy overlapping pitch ranges with closely spaced fundamentals, causing harmonics to overlap in time-frequency representations. Existing models typically use harmonic constant-Q transform (HCQT) inputs for frequency-adaptive resolution, but HCQT feature extraction is costly when training mixtures are generated on the fly. The authors compare HCQT with a linear short-time Fourier transform (STFT) whose frequency bins are directly fed to the model. Despite fixed frequency resolution and no pitch-aligned input grid, the linear STFT outperforms HCQT while substantially reducing feature-extraction cost. Further analysis shows that longer analysis windows or wider spectral coverage bring no additional gains, and restricting inputs to the predicted pitch range removes the linear STFT's advantage. The results suggest finer frequency resolution does not necessarily improve vocal ensemble multi-pitch estimation, and shorter analysis windows may better suit time-varying vocal pitches.

Paper Overview

Field: Machine Learning
Authors: Junyoung Koh, Hao-Wen Dong
Date: 2026-10-02
arXiv: 2610.03656

Abstract

Multi-pitch estimation in vocal ensembles is challenging because singers occupy overlapping pitch ranges and often sing at closely spaced fundamental frequencies, causing their harmonics to overlap in time-frequency representations. Existing models commonly use harmonic constant-Q transform (HCQT)-based representations to provide frequency-adaptive resolution, at the cost of expensive feature extraction when training mixtures are generated on the fly.

Key Points

The authors revisit this design and compare HCQT with a linear short-time Fourier transform (STFT), whose frequency bins are directly provided as model inputs.
Despite its fixed frequency resolution and the absence of a pitch-aligned input grid, the linear STFT outperforms HCQT while substantially reducing feature-extraction cost.
Further analysis shows that longer analysis windows or wider spectral coverage do not yield additional improvements.
Restricting the input to the predicted pitch range removes the linear STFT's advantage.

Conclusion

These results suggest that finer frequency resolution does not necessarily improve multi-pitch estimation in vocal ensembles, and that shorter analysis windows may be more effective for time-varying vocal pitches.

---

*Auto-collected on 2026-10-06*