Paper Overview
Field: NLP Authors: Zhaofeng Wu, Shiqi Wang, Boya Peng Published: 2026-04-22 arXiv: 2604.20835
Summary
Modern language models demonstrate impressive coding capabilities in common programming languages (PLs), such as C++ and Python, but their performance in lower-resource PLs is often limited by training data availability. In principle, however, most programming skills are universal across PLs, so capability acquired in one PL should transfer to others.
This work introduces the task of zero-shot cross-programming-language transfer for code RL. The authors find that, for Llama-3.1, RL training for code generation in a source PL fails to improve—and sometimes even degrades—performance on other target PLs.
Key points
- Problem: RL on one source language does not transfer (and can hurt) performance on unseen target languages.
- Hypothesis: Effective RL transfer requires a generalizable SFT initialization before RL.
- Method: Parallel-SFT, an SFT strategy that incorporates "parallel programs" (functionally equivalent code written in multiple programming languages) into the SFT data mix.
- Result: Parallel-SFT models show better generalization to unseen programming languages when subsequently trained with RL.
- Analysis: Model internal representations reveal that Parallel-SFT leads to a more functionality-centric latent space, where equivalent programs across PLs cluster more tightly—presumably contributing to improved transferability.