[论文] EmoRES-TTS: Residual-Enhanced Vector Steering for Emotional Speech Gen...

论文概要

研究领域: NLP 作者: Kuan-Po Huang, Haohe Liu, Puyuan Peng, Haibin Wu, Zhaoheng Ni, Hung-yi Lee, Jinwon Lee, Neha Chachra 发布时间: 2026-09-29 arXiv: 2609.38157

中文摘要

情感条件化的文本转语音（TTS）模型可能无法可靠地表达所请求的情感，而通过额外训练来改进可控性在计算和情感标注语音训练数据两方面成本都很高。因此，我们研究向量转向（vector steering）——一种免训练的方法，通过修改冻结模型的内部表示来控制输出。CoCoEmo是一种传统的情感TTS向量转向方法，将每个情感向量视为由单一全局强度控制的不可分方向，限制了对所请求情感的遵循度。本文首先发现，情感向量可以分解为一个将语音推离中性表达的共享分量和一个引导生成朝向所请求情感的残差分量。基于这一发现，我们提出EmoRES——情感残差增强TTS转向方法，无需重新训练骨干模型即可分别控制两个分量。在IEMOCAP上，EmoRES在IndexTTS-2和CosyVoice2两个骨干上的四项客观情感指标全面优于CoCoEmo。排名相关分别提升26.13和12.97个百分点，对应相对增益118.8%和33.1%；情感命中率分别提升12.95和6.92个百分点，对应相对增益20.1%和9.8%。人类评估进一步表明，听者对主导请求情感的识别率相对提升高达35.0%，保真度提升高达17.3%，在自然度上最多63.8%的成对比较中偏好EmoRES。分量消融进一步证明，有效的控制需要保留共享分量的同时增强情感转向向量的残差分量。

原文摘要

Emotion-conditioned text-to-speech (TTS) models may fail to express the requested emotion reliably, and improving controllability by additional training is costly in both computation and emotion-labeled speech training data. We therefore study vector steering, a training-free approach that modifies the internal representations of a frozen model. CoCoEmo, a conventional vector steering method for emotion TTS, treats each emotion vector as an indivisible direction controlled by a single global strength, limiting adherence to the requested emotion. In this work, we first discover that an emotion vector can be decomposed into a shared component that moves speech away from neutral expression and a residual component that directs generation toward the requested emotion. Building on this finding,...

#论文 #arXiv #NLP #小凯