Summary
As LLM pretraining shifts from compute-constrained to data-constrained regimes, Yu and Xiong (CMU) propose SynPro, a reinforcement learning framework that optimizes two operations—rewriting and reformatting—to produce diverse training tokens from the same organic data. The RL optimizer is driven by rewards across three dimensions: quality, faithfulness, and data influence. In data-constrained settings (only 10% of Chinchilla-optimal tokens), SynPro unlocks 3.7–5.2x more effective tokens than standard repetition, and at the 1.1B scale even outperforms a non-data-constrained control group, without causing distribution collapse. Open questions remain: whether rewriting and reformatting introduce repeated patterns or artifacts; how subjective quality rewards are adjudicated in the RL optimizer (who defines 'high quality'?); and how SynPro-generated data performs when mixed with other sources, such as human-written data. References include arXiv:2605.17849, Hoffmann et al. (2022) on compute-optimal training, and Muennighoff et al. (2024) on data-constrained scaling.
LLM pretraining is shifting from compute-constrained to data-constrained. The SynPro framework from Yu and Xiong (CMU) uses reinforcement learning to optimize two operations—rewriting and reformatting—to generate diverse training tokens from the same organic data. The RL optimizer is driven by rewards across three dimensions: quality, faithfulness, and data influence.
In data-constrained scenarios (using only 10% of Chinchilla-optimal tokens), SynPro unlocks 3.7–5.2x more effective tokens than standard repetition. At the 1.1B scale, it even surpasses the "non-data-constrained" control group. It does not cause distribution collapse.
Unclear points:
- Do rewriting and reformatting introduce repeated patterns or artifacts into the data?
- Subjectivity of quality rewards in the RL optimizer—who judges what counts as "high quality"?
- How does SynPro-generated data perform when mixed with other generation approaches (e.g., human-written data)?
References
1. Yu, Z., & Xiong, C. (2026). *Generating Pretraining Tokens from Organic Data for Data-Bound Scaling*. arXiv:2605.17849 [cs.CL].
2. Hoffmann, J., et al. (2022). *Training Compute-Optimal Large Language Models*. NeurIPS.
3. Muennighoff, N., et al. (2024). *Scaling Data-Constrained Language Models*. NeurIPS.
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177620397