Agogic: How a 0.8B Music Model Beat a 27B Model by Changing Tokenization

A 0.8B-parameter music generation model outperformed a 27B model from the same family not through more data, longer training, or cleverer architecture, but by changing how music is encoded into tokens.

This is the core finding of the Agogic paper, and it challenges one of the AI field's most entrenched beliefs: scale is all you need.

A Variable That Got Skipped

The text-to-music pipeline is long: pick a backbone, prepare data, design a training recipe, choose a decoding strategy—and then, almost as an afterthought, convert music into tokens. This last step is usually a default nobody seriously tests.

Agogic's approach is simple: freeze every other variable and only swap tokenization.

• Backbone: Qwen3.5, ranging from 0.8B to 27B
• Data: identical
• Training budget: identical
• Decoding: identical
• The sole variable: tokenization

They tested seven tokenizations. The results were unexpectedly clean.

Representation, Not Model Size, Is the Bottleneck

Key numbers:

• PMT (a new representation introduced in the paper: 10 ms timing precision, per-note dynamics, multi-track texture, 609 symbols): 0.8B model achieves FMD = 159
• Beat-grid tokenizations (REMI and its descendants, the current mainstream): 0.8B model achieves FMD = 272–286

FMD (Fréchet Music Distance) is better when lower. PMT is 1.7–1.8× lower than beat-grid methods, and up to 2.8× lower under some protocols.

In plain language: a 0.8B model with PMT beats a 27B model from the same family using beat-grid tokenization.

Scaling the backbone 34× barely budged FMD. Swapping tokenization roughly halved it.

The paper's exact phrasing is: *"Representation, not model size, is the bottleneck variable for distribution fidelity."*

Why Beat-Grid Tokenization Loses

Current mainstream music tokenization splits into two camps, each with its own compromise:

Beat-grid camp (REMI and descendants): aligns every note's onset to beat-grid positions. The benefit is structural stability. The cost is that quantization discards micro-timing and per-note dynamics—exactly the elements that distinguish a real performance from a mechanical score.

Text/ABC camp: encodes music as text-format scores (e.g., ABC notation). More complete information, but poor alignment with the natural-language pretraining of LLMs.

PMT takes a third path: performance resolution—10 ms timing precision, per-note dynamics, multi-track texture. It does not record *what the score looks like*; it records *how this specific performance was actually played*.

The distinction matters: the same piece sounds dramatically different depending on the performer. Beat-grid methods discard those differences. PMT preserves them.

Not Luck, but a Category Effect

To rule out "PMT just happens to have a lucky vocabulary," the paper ran three validation experiments:

1. Swap in a smaller backbone. A 26M-parameter from-scratch model still favors PMT. The effect is not specific to large models.

2. Swap in a different performance-resolution tokenizer. The effect persists. It is not unique to PMT; it belongs to the performance-resolution category of representations.

3. Not just a side effect of finer grids. When PMT-generated onsets are aligned back down to beat-grid resolution, PMT still leads by 67–129 FMD. Higher resolution alone does not explain the win—the representation itself is better.

A Troubling Byproduct: The Imprinting Effect

The paper also surfaces an unsettling phenomenon called the imprinting diagnostic:

> Published text-to-MIDI systems produce nearly identical chord-time distributions on two completely unrelated domains—72% vs 71%.

In plain language: whatever prompt you give it, the model largely does not care. It produces whatever is closest to its training distribution.

This contradicts the surface impression that current text-to-music models are following prompts. They appear to listen; in reality, they replay what they have already memorized. The finding echoes a broader suspicion that current LLMs "understand" less deeply than they appear to.

A Lightweight Patch

The good news: the paper identifies a cheap fix—decode-time constraint.

Without it: instrument F1 = 0.28, correct key = 0.16. With it: instrument F1 = 0.60 (more than doubled), correct key = 0.35 (more than doubled), with no loss in distribution quality.

This means prompt-following and distribution fidelity are separable problems. You can fix one without touching the other.

What This Means

Agogic's conclusion is a cross-domain engineering principle: before scaling up, identify which variable is actually the bottleneck.

The AI community's dominant faith is that scale solves everything. Agogic offers a counterexample: on music generation, the bottleneck is representation, not model size. A 34× scale-up barely helps. A better representation roughly halves the error.

The lesson applies to any multi-variable system: run a controlled ablation, find the bottleneck, then decide where to spend resources. Blind scaling may simply be the most expensive way to optimize the wrong knob.

The paper also releases a full evaluation framework, 25+ checkpoints, and two corpora (86.6k aligned caption/MIDI/ABC/audio pairs, plus 6.25M captioned items—the largest annotated music corpus to date). Any future music-representation proposal can now be measured, not just claimed.

That may be the paper's biggest contribution: it turns representation choice from a belief into a measurable quantity.

---

Paper link: https://arxiv.org/abs/2608.03999

Code & data: the paper announces release of the evaluation framework, 25+ checkpoints, and two corpora.