YuE2: Unifying Symbolic Scores and Audio Waves in One Checkpoint
> Project: multimodal-art-projection/YuE > Stars today: +500 > Language: Python > Paper: arXiv:2609.33757 > Institutions: HKUST, M·A·P, Tokenwave.AI, NYU, Stanford, MBZUAI, NOIZ, ACE Studio > Positioning: Unified symbolic and audio music generation
---
A Scenario
You give Suno a set of lyrics and a style description, and it gives you a song. It sounds fine. But you want to change the chorus melody — you can't. Suno gives you finished audio, not a score. To change anything, you can only regenerate and hope it turns out better this time.
YuE2 takes a different approach: it first gives you a readable, editable score (melody + chords), then renders a full song (vocals + accompaniment) from that score. Want to change the chorus melody? Just edit the score, and YuE2 re-renders from the modified version.
Generation is a black box, but everything before generation is a white box.
---
Core Numbers
On WildSongBench, evaluated over 192 prompts:
- YuE2 (best-of-8) achieves a 6.9632 SongBench Avg — the highest mean across all evaluation settings
- Competitive with Suno v5/v6
- #3 on the global HuggingFace models trending chart (Sep 17, 2026)
- #1 on the HuggingFace Text-to-Audio trending chart (Sep 20, 2026)
- Read: inspect melodic contours and chord progressions
- Edit: change the chorus melody, swap chords
- Re-render: regenerate audio from the edited score
- "Change the last chord of the chorus to a dominant seventh"
- "Add a transition to the second verse"
- "Lower the whole song by one key"
- MIDI-derived synthetic supervision
- Structured decoding
- Autoregressive distillation
- Evaluation on 8 benchmark sets
- SongBench Avg: overall quality mean
- Song Quality: audio quality
- Text Alignment: lyric alignment
- Musicians / arrangers: want AI-assisted creation without losing control over melody and chords
- Content creators: need custom background music with iterative revisions
- Music educators: can generate teaching materials with readable, editable scores
- AI music researchers: interested in unified symbolic + audio generation
- Casual users who just want one-click songs (Suno is more convenient)
- Teams needing API integration into products (YuE2 is a model, not a service)
- Hardware-limited users (the 3B-parameter model needs a GPU)
This isn't "approaching commercial quality" — it's already at the frontier.
---
Core Innovation: Symbolic Planning
YuE2's key design is symbolic planning, a two-stage pipeline:
Step 1: Symbolic Planning
Input lyrics + style prompt → YuE2 generates a melody + chord plan.
The plan is readable: you can see the melodic contour of the verse, the chord progression of the chorus, key changes in the bridge. This isn't audio features — it's musical notation, like a lead sheet a composer would write.
Step 2: Audio Rendering
On top of this plan, YuE2 renders a complete song: vocals sing the melody, accompaniment realizes the chords, drums follow the style.
Key point: both stages run on the same checkpoint. Not two models — two phases of one model.
---
Why "Unification" Matters
Music generation has had two historical routes:
| Route | Examples | Pros | Cons | |------|-----------|------|------| | Symbolic generation | MusicTransformer, SheetSage | Readable, editable, musically structured | Sounds like MIDI, unrealistic | | Audio generation | Suno, Udio, MusicGen | Sounds like real music | Not editable, black box |
These routes have long been split: symbolic researchers dismissed audio work as "lacking musicality," while audio researchers dismissed symbolic work as "sounding like toys."
YuE2's paper title is "Unifying Symbolic and Audio Music Generation at Frontier Quality" — the unification means you get both editable symbols and frontier-quality audio. No longer an either/or choice.
---
Three Derived Capabilities
Symbolic planning enables three capabilities traditional audio generation cannot offer:
1. White-Box Generation
Every generated song has a corresponding "score" that you can:
This turns "AI generation" from "rolling dice" into "iterative creation."
2. Zero-Shot Covers
Give YuE2 an existing song's audio → it transcribes it to a score (using SheetSage2) → you specify a new style → YuE2 re-renders the original melody in the new style.
A folk song becomes an EDM track — same melody, entirely new arrangement. No fine-tuning, no training required.
3. Agent Editing
YuE2 ships an agent skill that lets AI agents modify songs conversationally:
The agent reads the score, edits it, and triggers re-rendering. Music editing becomes a conversation.
---
SheetSage2: The Infrastructure for Covers
YuE2's cover capability depends on SheetSage2, a music transcription model released on arXiv Oct 6, 2026 (arXiv:2610.05336).
SheetSage2 features:
Without SheetSage2, YuE2's covers wouldn't be possible — you need to convert existing songs into symbols before editing them. Transcription is the prerequisite for covers.
---
Evaluation: WildSongBench
YuE2 is evaluated on WildSongBench, a public benchmark with 192 prompts covering multiple styles and languages.
Evaluation dimensions:
YuE2's best-of-8 setting reaches a 6.9632 average — the highest mean observed across all evaluation settings, including commercial systems.
YuE2 is also running a public blind listening study (Music Arena) where users anonymously compare clips from YuE2 and leading commercial systems, choosing A, B, or tie. This open-evaluation stance is rare among academic projects.
---
How It Differs from Suno
| Dimension | Suno v5/v6 | YuE2 | |------|-----------|------| | Generation | End-to-end audio | Symbolic planning + audio rendering | | Editability | None (finished audio) | Yes (editable score) | | Cover capability | None | Zero-shot covers | | Conversational editing | None | Agent skill | | Deployment | Cloud service | Open source, local | | Model openness | Closed | Open (HuggingFace: m-a-p/YuE2-3B) | | Quality level | Frontier | Frontier (competitive) |
Suno is a "black-box finished product"; YuE2 is "white-box creation." The former is a restaurant (order and get served); the latter is a kitchen (you cook, but with full tools).
---
A Deeper Observation
YuE2's symbolic planning points to a broader trend: the next battleground for AI generation is "editability."
From 2023–2025, the competition was "who generates better" — higher quality, longer duration, more styles. In 2026, the competition is "who gives users more control."
This trend has already happened in image generation (ArtCraft uses 3D scenes to control composition) and is now happening in music (YuE2 uses symbolic planning to control melody and chords). Models are converging; the control layer is differentiating.
When Suno, Udio, and MusicGen can all generate high-quality songs, the deciding factor is no longer "who sounds better" but "can I precisely get the music I want." That's the bet YuE2 is making.
---
Who It's For
Not for:
One-Sentence Takeaway
YuE2 turns music generation from a "black-box product" into "white-box creation" — you first get a readable, editable score, then render a frontier-quality song from it. While everyone else competes on "who generates better," YuE2 competes on "who gives users more control." Unifying symbols and audio isn't just a technical breakthrough — it's a shift in the creative paradigm.