Imagine placing two newborns in an empty room — no books, no language input, no human knowledge. One generates problems; the other guesses. That is essentially what researchers at Stanford and Tel Aviv University did: two randomly initialized Transformers, zero training data. They not only learned to predict natural language text but also spontaneously discovered the Fibonacci sequence, geometric sequences, and quadratic/cubic polynomial sequences.
Paper: Self-Play Pretraining with Zero Data (arXiv: 2609.30063)
Core question: is data the bottleneck, or compute?
Current pretraining can be summarized as more data + more compute = better models, with a hidden assumption: data must be produced, cleaned, and labeled by humans. The radical question: what if models could generate their own training data?
Not distillation from a large model, not GPT-4 training GPT-5 — but starting from random initialization, two models playing against each other: one creating data, one learning from it. If that works, the pretraining bottleneck shifts from "human knowledge" to "compute." This paper is the first proof-of-concept.
Method: a generator–learner self-play loop
The architecture is remarkably simple, with two roles:
Generator: a Transformer that writes programs in a Brainfuck-like Turing-complete language — only 8 basic instructions (move pointer, increment/decrement byte, read/write, loops), yet capable of computing anything computable.
Learner: an identical Transformer that predicts the byte streams produced by programs, trained with standard next-token prediction (cross-entropy).
Each self-play round:
1. The Generator samples N programs 2. Each program executes on a universal Turing machine with a random input tape, producing byte sequences 3. The Learner takes one gradient step on those sequences 4. The Generator takes one policy-gradient step based on the "learning progress reward"
Step 4 is the key. The reward is not "minimize the Learner's loss" — that would produce only trivial programs. The reward is learning progress: programs must sit exactly at the Learner's capability frontier. Too easy, no progress; too hard, no progress either.
It's like a good teacher setting problems: not too simple, not too hard — right in the student's zone of proximal development.
Three-tier program pool
Each round, the Generator doesn't only sample from scratch. The pool combines:
- Fresh: newly sampled programs, for global exploration
- Mutation: variants of previously high-reward programs, for local refinement
- Replay: programs from earlier rounds, preventing catastrophic forgetting
- Initially: predicting the most common byte (prior)
- After 2–3 examples: starts copying earlier bytes
- After 4 examples: computes low-nibble addition correctly
- After 8 examples: computes high-nibble addition correctly
- Then: locks in the correct strategy with rising confidence
This mirrors AlphaGo's self-play: balancing new strategy exploration against consolidating learned strategies.
Results: zero-shot scaling on natural data
The most surprising part. The two models never saw any natural data during training. Evaluated zero-shot (no gradient updates) on natural data:
Across text (DCLM), images, raw audio, music, DNA, formal mathematics, and C source code, zero-shot loss falls as a power law with self-play compute.
Even more striking: the self-play scaling exponent is comparable to that of direct natural-data training.
Implication: a large part of why natural-data pretraining works may not be teaching the model "facts about the world" (contingent information) but teaching it "universal predictive structure" — and the latter can be generated from compute via self-play.
Emergent mathematical structure
The Generator spontaneously discovered sequences known to human mathematics:
| Sequence type | Example program | Output | Pool frequency |
|---------|---------|------|-----------|
| Fibonacci | S,[[.C&>.C>] | 1,1,2,3,5,... | 512 |
| Geometric | S+[.L>] | 1,3,9,27,81,... | 256 |
| Quadratic | S,.[<C>>VX<RX++] | 9,25,59,111,... | 512 |
| Cubic | S+[[-.L>L>-]-] | 0,254,236,74,... | 512 |
None of this was hardcoded — the Generator "discovered" these structures by RL exploration from random initialization.
Emergence of in-context learning (ICL)
Self-play pretrained models also developed in-context learning — inferring rules from few-shot examples on unseen tasks.
On a SUM task (4-bit low-nibble addition on bytes), the model's behavior evolved interestingly:
Theory: the universal data hypothesis
The paper decomposes natural data into two components:
Contingent Information (\(D_c\)): facts about this particular world. "Paris is the capital of France," "water boils at 100 degrees." Only obtainable through interaction with the world.
Universal Predictive Structure (\(D_u\)): predictive regularities shared across domains. Patterns repeat, conditional distributions have structure, information is compressible.
Traditional scaling law:
The paper refines it to:
In self-play pretraining, \(D_c\) is fixed at zero (no natural data); only \(D_u\) grows. If \(D_u(T) ∝ T^η\) with self-play compute T:
This explains the power-law loss decay — self-play grows universal predictive structure, which is a major component of natural-data scaling laws.
Self-play as a pretraining accelerator
A practical finding: self-play pretraining accelerates subsequent natural-data training. Starting natural-data training from a self-play checkpoint converges faster than from random initialization. Self-play gives the model a "universal structure foundation," making later knowledge acquisition more efficient — like learning math and logic first, then specific subjects.
Engineering insights
1. "Creating problems is harder than solving them," AI edition. The Generator trains far harder than the Learner. The Learner does supervised next-token prediction; the Generator must search a vast program space with RL — hence the three-tier pool design (fresh + mutation + replay); pure RL is too unstable and needs anchoring.
2. Automating the zone of proximal development. Vygotsky's theory is quantified as the "learning progress reward" — automatic curriculum design without human-defined difficulty gradients.
3. Turing completeness as an inductive bias. Using a universal Turing-machine language like Brainfuck introduces no domain-specific bias; any computable data-generating process is representable. Minimal-inductive-bias at its extreme.
4. Scientific value of the tabula rasa setting. The authors stress the blank-slate setup is a "controlled scientific experiment," not the most practical pretraining method. In practice self-play can start from existing checkpoints. The key scientific question: can self-generated experience keep scaling the frontier after natural data is exhausted? The answer is yes.
Personal reflections
This paper raises a deep philosophical question: what is learning?
Traditionally, learning extracts information from data. This paper shows a different kind: extracting structure from computation. Two randomly initialized models, through self-play, discovered the Fibonacci sequence — a mathematical concept humans took millennia to formalize.
This suggests computable structures are not invented but discovered. They exist in the space of all computable functions, waiting for any sufficiently clever search process. Self-play is such a process.
More radically: perhaps human intelligence has a similar mechanism. An infant's brain isn't an empty hard drive awaiting data; it has an internal generate-and-learn loop — infants generate their own stimuli (manipulating objects, vocalizing) while learning their regularities. Human language, math, and music may all be products of this inner loop.
The paper's title quotes Wheeler: "So much from so little, almost everything from almost nothing." It captures the core finding: from almost nothing (random initialization + a Turing machine), self-play can generate nearly all universal predictive structure.
Limitations remain explicit: models under 25M parameters, 4K context. Whether this holds at scale is open. But as a proof-of-concept it is striking — it demonstrates "data is not the only bottleneck in pretraining; compute can be."
When high-quality internet text runs out, self-play pretraining may become the next scaling dimension: not bigger datasets, but deeper self-search.
---
Paper: arXiv:2609.30063 Authors: Aditya Cowsik, Kfir Dolev, Michael Y. Li, G. Bruno De Luca, Nourya Cohen, Noah D. Goodman, Yoav Levine Institutions: Stanford University, Tel Aviv University, LAPTh/USMB