Recreating AlphaGo on a $10,000 Budget: Eric Jang's Sabbatical Project and What It Reveals About AGI

Former DeepMind scientist Eric Jang reproduced a strong Go bot from scratch during his sabbatical using roughly $10,000 of compute donated by Prime Intellect (about $3,000 of which went to final training), deployed online for humans to play against. The most alarming part of the project is not the price tag—it's what it reveals: AlphaGo's reinforcement learning is, from start to finish, supervised learning.

1. A Decade of Compute Democratization

• 2016: DeepMind beats Lee Sedol with AlphaGo—3×10²³ FLOPs, a massive outlier on the compute curve, built by a full research team.
• 2020: Jane Street's David Wu releases KataGo, cutting training compute for a strong Go AI by 40×.
• 2026: Eric Jang, on sabbatical, reproduces a strong bot from scratch for ~$10K.

Jang used Claude Opus as his coding assistant, even writing a /experiment skill that let the AI propose hypotheses, run experiments, compile charts, and write reports.

2. MCTS: RL Disguised as Supervised Learning

AlphaGo combines three components:

• Policy network: 361-point move probability distribution ("intuition")
• Value network: win-rate estimate of the current position
• MCTS: builds a search tree at decision time via simulated play

The counterintuitive truth Jang highlights:

> "AlphaGo is strong not because search is deep or networks are large, but because MCTS generates a 'playing here would be better' improvement label for every single move, turning the entire RL problem into repeated supervised learning."

The training loop:

1. Play games with the current policy network + MCTS to generate data 2. After each search, MCTS produces an improved move probability distribution for every position 3. Train the policy network with cross-entropy loss to imitate MCTS's decisions 4. Train the value network on final game outcomes

The policy network's target is "mimic MCTS," not raw game rewards. This sidesteps the notorious credit assignment problem: unlike RLHF/PPO—where a model must figure out which of 300 tokens deserves blame for a scalar reward—MCTS gives a dense, local, supervisable improvement signal for every move. Every iteration of RL is supervised learning.

AlphaGo Zero simplified further: no human games, pure self-play, a merged two-head ResNet predicting policy and value. But the core remained—MCTS is the mentor that is always slightly stronger than the model.

3. ResNet's Local Inductive Bias Beats Transformers on a Budget

Jang's contrarian finding: in small-data / limited-compute regimes, ResNet still outperforms Transformers.

• CNN/ResNet inductive bias: locality + translation equivariance
• Transformer inductive bias: global attention, no structural priors

Go is highly local—liberties, eyes, life-and-death, connections are local geometric problems on a symmetric 19×19 board. KataGo even found that adding global feature pooling to ResNet gives significant gains: extract local features first, then lightly aggregate globally—much like how humans read a board.

But Jang concedes that at sufficient scale, Transformers win. Human-designed biases give short-term efficiency; more general methods eventually triumph (the Bitter Lesson)—the threshold for "eventually" is just higher than many assume.

4. Why LLMs Cannot Copy AlphaGo: Three Structural Bottlenecks

Action-space dimensionality. Go has 361 discrete, structured moves. LLMs have 32K–200K vocab actions per token, and language lacks a compact state representation for evaluating "position value." Designing an MCTS equivalent for open-ended text is extremely expensive.

Sparse rewards and credit assignment. Go's MCTS gives per-move improvement signals; LLM rewards are sparse and delayed (+1 if the final answer passes, 0 otherwise). PPO trains a critic with fragile actor–critic coupling; GRPO spreads the whole-trajectory reward evenly across tokens; research like TEMPO (prefix trees, per-branch credit) and VinePPO/PRMs is early-stage.

Bits-per-FLOP crisis. Most LLM tokens in a reasoning trace are filler—connectives, repetitions, formatting—with near-zero information toward the outcome. LLM RL thus gets far fewer learnable bits per FLOP than Go, where every simulation yields signal.

5. The Blind Spot Beyond the Bitter Lesson

Sutton's Bitter Lesson assumes abundant data and verifiable feedback loops. Go satisfies this perfectly: clear rules, perfect information, instant unambiguous verification, infinite self-play. The real world does not: scientific experiments take months or years, business success is ambiguously defined, creative work lacks objective scoring. Outer-loop validation is the fundamental divide—Go closes a hypothesis→test→feedback loop in milliseconds; reality's feedback can outlast any learner.

Jang also notes AGI requires lateral thinking—cross-domain analogy, discovering hidden connections, inventing new questions—which no current system reliably exhibits. AlphaGo's "creative" moves were discoveries inside a closed, fixed-rule space.

On scaling laws:

> "Scaling laws only apply once the recipe already works and data quality is good. You can't simultaneously figure out 'how to do it right' and 'how to scale' and expect the scaling law to tell you the answer."

Scaling laws are descriptive, not prescriptive. Blindly piling on compute before the problem is well-defined is expensive futility.

6. Engineering Takeaways

• Simplification: 2026 GPUs allow replacing AlphaGo Zero's distributed async RL + replay buffer architecture with a simple synchronous loop (collect data → supervised train → repeat); auxiliary supervision targets are no longer necessary.
• Small-board pretraining: train on 9×9 first to solve the endgame value function, then transfer to 19×19, compressing the warm-up phase where AlphaGo Zero spent its first 30 hours catching up to supervised baselines.
• Best-response training: Jang's online bot used best-response training against a KataGo model for strong initialization. Learning from a strong mentor beats fumbling from zero—a logic consistent with human apprenticeship.

7. Conclusion

1. Compute democratization is real, but deep understanding matters more than low cost 2. AlphaGo's elegance lies in turning RL into supervised learning via MCTS's policy improvement operator 3. LLM RL struggles are structural: oversized action spaces, sparse rewards, intractable credit assignment 4. The Bitter Lesson has scope conditions—verifiable outer loops—that the open world fails 5. Scaling laws cannot substitute for understanding

Sometimes, as Jang says, the best way to understand the future is to go back and rebuild the past by hand.

References

• Eric Jang, Dwarkesh Podcast: "Building AlphaGo from scratch" (2026-05)
• David Wu, "KataGo" (2020)
• Silver et al., "Mastering the game of Go without human knowledge" (AlphaGo Zero, Nature 2017)
• Silver et al., "Mastering Chess and Shogi by Self-Play" (AlphaZero, 2017)
• Rich Sutton, "The Bitter Lesson" (2019)
• Andy Jones, "Scaling Scaling Laws with Board Games" (2021)
• Sutton et al., "Reinforcement Learning: An Introduction"