Reinforcement Learning Teaches LLMs to Translate Unseen Languages: Learning How to Learn, Not Memorizing

Forum topic · 小凯 · 2026-06-07

Summary

Researchers at the University of Zurich and ETH Zurich show that large language models trained with reinforcement learning (RL) translate unseen, low-resource languages better than models trained with supervised fine-tuning (SFT), even though the RL models never saw those languages during training. The team frames translation of unseen languages as a meta-learning problem: models must learn to leverage in-context resources such as grammar sketches and dictionaries. Using 14 low-resource training languages, GRPO with chrF as reward, the RL-trained model substantially outperforms SFT on unseen test languages like Kalamang. A key ablation shows SFT models barely use the provided context, while RL models rely heavily on it, demonstrating that RL elicits genuine contextual leveraging rather than memorization. The work has implications for endangered-language preservation, suggesting translation tools can be built from linguistic field documentation alone.

Imagine being handed a grammar book written in a completely unknown language—unfamiliar symbols, strange inflection rules, and a dictionary with hundreds of words. You get half an hour, then you must translate a passage into English.

This is the test that researchers at the University of Zurich and ETH Zurich gave to large language models. Their surprising finding: models trained with reinforcement learning (RL) translate unseen languages better than models trained with supervised fine-tuning (SFT)—even though the RL models never encountered these languages during training.

The Problem: Memorization vs. Learning to Learn

There are two mainstream approaches for low-resource translation with LLMs:

Route 1: Continued training. Feed the model parallel corpora in a target language so it "memorizes" it. Problem: overfitting—performance drops on languages never seen.

Route 2: In-context learning. Put the grammar book and dictionary in the prompt and let the model "look things up" on the fly. Problem: models often don't actually read these resources—prior studies found LLMs rely mainly on parallel examples and largely ignore grammatical descriptions.

Both routes share the same flaw: the model memorizes specific languages rather than learning how to use linguistic resources.

Method: Teaching Models "How to Learn" with RL

The researchers' core insight: translating unseen languages is fundamentally a meta-learning problem. What models need is not vocabulary and grammar of specific languages, but a meta-skill—contextual leveraging: given a grammar and a dictionary, how to effectively extract and apply that linguistic knowledge.

The approach is surprisingly simple:

1. Training data: 14 low-resource languages, including 8 grammars-based languages and 6 Romansh variants. Each training sample contains dictionary entries (~2 per source word), 3–5 parallel sentence pairs, and a grammar excerpt (~2,800 tokens).

2. RL training: The chrF translation metric serves as the reward signal, optimized with the GRPO algorithm. The model sees the grammar and dictionary, attempts translation, and is rewarded for quality.

3. Key design: Training uses only "seen languages"; testing uses "unseen languages"—including Kalamang (a Papuan language) and 4 OOD languages the model never encountered.

Results: RL Generalizes, SFT Doesn't

On seen languages, SFT is stronger. Unsurprising—SFT trained directly on those languages.
On unseen languages, RL crushes SFT. This is the key finding: RL-trained models translate never-seen languages significantly better.

Ablation reveals the mechanism. When researchers removed the context (dictionary + grammar) at test time:

SFT models barely changed—meaning they weren't using the context at all, relying on memorized patterns.
RL models dropped sharply—meaning they genuinely exploited the linguistic knowledge in context.

This ablation is the most elegant part of the paper: it directly shows that RL training teaches models to "read the grammar book," while SFT teaches them to "memorize answers."

Why RL Succeeds Where SFT Fails

SFT's training signal is "imitate the correct translation." The laziest strategy is memorizing patterns from training languages, not understanding the grammar book—reading it requires reasoning; memorization doesn't.

RL's training signal is "translation quality." To earn high reward, the model must use the context—since test languages differ from training languages, rote memorization doesn't work. RL forces the model to develop genuine linguistic comprehension.

This mirrors RL's success in mathematical reasoning: RLVR (Reinforcement Learning with Verifiable Rewards) works not by making models memorize more answers, but by teaching them reasoning processes. This paper extends that same logic to language learning.

The Bigger Picture

The work raises a deeper question: what training method gives models transferable capabilities instead of rote memorization?

The answer seems to be: give the model an objective (translation quality) and let it explore how to use available resources (grammar + dictionary), rather than telling it the answers (SFT). This strikingly echoes human learning—cramming produces exam machines; inquiry-based learning produces real understanding.

For endangered-language preservation, the implications are practical. Of the world's ~7,000 languages, most lack enough parallel data to train translation systems. But if models can learn to "translate from a grammar book," then linguists' field documentation alone could be enough to build translation tools for these languages.

---

Paper: Reinforcement Learning Elicits Contextual Learning of Unseen Language Translation

Code: github.com/hanxuhu/rl-new-language

Authors: Hanxu Hu, Zdeněk Šnajdr, Pinzhen Chen, Jannis Vamvas, Rico Sennrich (University of Zurich, ETH Zurich, Queen's University Belfast)