LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling

Forum topic · 小凯 · 2026-07-01

Summary

LeVo 2 (arXiv:2507.00002) is a hybrid LLM-Diffusion framework for controllable full-length song generation, addressing a structural trade-off in language-model-based music systems: mixed-token modeling preserves vocal-instrument coordination but hides track-specific details, while dual-track prediction improves acoustics but weakens global planning. LeVo 2 treats this as hierarchical modeling: LeLM first predicts mixed tokens for semantic planning, then predicts vocal and accompaniment tokens in parallel for track-specific refinement, with a diffusion-based Music Codec reconstructing full-length waveforms. The extended version introduces an aesthetic-guided alignment training scheme: automated music aesthetic evaluation assigns musicality-level conditions during pretraining, followed by progressive post-training with SFT, large-scale offline DPO, and closed-loop semi-online DPO to improve quality, controllability, and musicality. Modular extensions train track-specific LMs for acoustic refinement while preserving the aligned semantic planner. Expert listening tests show LeVo 2 outperforms open-source baselines on six subjective dimensions and approaches leading commercial systems on several metrics.

Paper Overview

Field: Audio Generation Authors: Shun Lei, Huaicheng Zhang, Dapeng Wu arXiv: 2507.00002

Abstract

Full-length song generation must preserve coherence and musicality, render detailed vocal and accompaniment acoustics, and follow lyrics and prompts. Existing language model-based systems face a structural trade-off: mixed-token modeling preserves vocal-instrument coordination but obscures track-specific details, whereas dual-track prediction improves acoustics but requires longer sequences and weakens global planning. We present LeVo 2, a hybrid LLM-Diffusion framework for controllable full-length song generation. LeVo 2 formulates this trade-off as hierarchical modeling: LeLM first predicts mixed tokens for semantic planning, then predicts vocal and accompaniment tokens in parallel for track-specific refinement, while a diffusion-based Music Codec reconstructs full-length waveforms.

Key Contributions

The core contribution of this extended version is an aesthetic-guided alignment training scheme:

Pretraining: An automated music aesthetic evaluation framework assigns musicality-level conditions to large-scale data, providing musicality priors before preference alignment.
Progressive post-training: Applies SFT, large-scale offline DPO, and closed-loop semi-online DPO to improve generation quality, controllability, and musicality, respectively.
Modular extension: Trains track-specific LMs for acoustic refinement while preserving the already-aligned semantic planner.

This scheme decouples musicality learning, controllability alignment, and acoustic refinement, mitigating optimization conflicts and the limitations of static offline preference pairs.

Results

Expert listening tests and objective evaluations show that LeVo 2 outperforms open-source baselines across six subjective dimensions and approaches leading commercial systems on several listening metrics.

--- *Auto-collected on 2026-07-01*

> Paper link: https://arxiv.org/abs/2507.00002