Recreating AlphaGo for $10K: Eric Jang's Sabbatical Project and What It Reveals About AGI

Forum topic · 小凯 · 2026-05-26

Summary

Former DeepMind scientist Eric Jang reproduced a strong Go-playing bot from scratch during a sabbatical, using roughly $10,000 of donated compute from decentralized platform Prime Intellect—of which only $3,000 went to final training—versus the team-scale effort and millions of dollars DeepMind spent in 2016. The deeper insight from the project is that AlphaGo's reinforcement learning is, at its core, iterated supervised learning: MCTS acts as a policy improvement operator, generating a dense, per-move improvement label for every board state, thereby sidestepping the credit assignment problem that plagues LLM post-training. The project also found that ResNets with local inductive biases outperform Transformers under small-data, low-compute regimes, while acknowledging Transformers win at scale. The article analyzes why LLMs cannot directly adopt the AlphaGo recipe—vast action spaces, sparse and delayed rewards, and low bits-per-FLOP—and argues that Sutton's Bitter Lesson applies only where outer-loop validation is cheap, a condition the open-ended real world does not meet.

Former DeepMind scientist Eric Jang used roughly $10,000 of compute (with only ~$3,000 for the final training run), donated by the decentralized compute platform Prime Intellect, to reproduce a surprisingly strong Go bot from scratch during his sabbatical—and deployed it online for anyone to play against.

The project's most striking takeaway is not the cost, but what it reveals: AlphaGo's reinforcement learning is, from start to finish, supervised learning.

A Decade of Compute Democratization

2016: DeepMind's AlphaGo beats Lee Sedol using a full research team and millions of dollars of compute (3×10²³ FLOPs).
2020: Jane Street researcher David Wu releases KataGo, cutting the training compute for a strong Go AI by ~40x.
2026: Eric Jang, alone, on cheap cloud servers, reproduces a strong bot and ships it as a playable service.

Jang used Claude Opus as his coding assistant, even writing a /experiment skill that let the AI propose hypotheses, run experiments, compile charts, and write reports.

MCTS: RL Disguised as Supervised Learning

AlphaGo combines a policy network (move intuition), a value network (position evaluation), and MCTS (search). The counterintuitive core: MCTS acts as a policy improvement operator.

The training loop:

1. Play games with the current policy network + MCTS 2. MCTS search produces an improved move distribution for every position 3. Train the policy network via cross-entropy to imitate MCTS's improved choices 4. Train the value network on final game outcomes

The policy network never learns directly from raw win/loss rewards. Instead, MCTS generates a dense, per-move improvement signal—"here is a better move than your intuition suggested." This eliminates the notorious credit assignment problem (which of 300 tokens deserves credit for a final reward?), which is why AlphaGo's training is remarkably stable.

AlphaGo Zero further removed human game records, using pure self-play, a ResNet backbone, and a shared two-headed network—but the policy improvement operator logic remained unchanged.

ResNet vs. Transformer on a Budget

Jang found that in small-data, limited-compute regimes, ResNet still outperforms Transformers. Go's core concepts (liberties, eyes, life-and-death, connections) are local geometric problems on a board with perfect rotational/mirror symmetry—exactly matching convolution's inductive biases. KataGo's addition of global feature pooling suggests a design philosophy: extract local features first, then apply lightweight global aggregation.

But Jang concedes Transformers win at scale—the Bitter Lesson, with a caveat: the "sufficient compute" threshold is higher than many assume.

Why LLMs Can't Copy the AlphaGo Recipe

Three structural bottlenecks:

1. Action space explosion: Go has 361 discrete moves; LLMs face 32K–200K vocabularies per token, with no compact state representation for search. 2. Sparse rewards and credit assignment: LLM rewards arrive only at the end of long trajectories. PPO's critic training is fragile; GRPO averages rewards across all tokens, ignoring branch points. Research like TEMPO (prefix-tree credit) and VinePPO (process reward models) is still early. 3. Bits-per-FLOP crisis: Most LLM tokens are filler with near-zero information about the outcome, so much of the RL compute is wasted.

Limits of the Bitter Lesson

Sutton's scaling thesis assumes cheap outer-loop validation. Go offers perfect information, instant verification, and unlimited self-play data. Scientific research, business decisions, and creative work have feedback loops measured in months or years, with ambiguous success criteria—conditions where scaling laws alone cannot help.

Jang's own conclusion: "Scaling laws only apply once the recipe already works and data quality is good. You can't simultaneously figure out how to do something right and expect scaling to tell you the answer."

Practical Engineering Takeaways

Modern GPUs allow synchronous, simple training loops, replacing AlphaGo Zero's distributed async RL + replay buffer architecture.
Small-board pretraining (9×9 first, then 19×19) compresses the expensive warm-up phase.
Best-response training against a strong opponent (KataGo) provides a far more efficient initialization than learning from zero.

Closing

Jang's project is a mirror for AI research: compute democratization is real, AlphaGo's elegance lies in converting RL into iterated supervised learning, LLM post-training's difficulties are structural, and the Bitter Lesson has applicability limits. As Jang puts it:

> "Sometimes the best way to understand the future is to go back and rebuild the past with your own hands."

References

Eric Jang, Dwarkesh Podcast: "Building AlphaGo from scratch" (2026-05)
David Wu, "KataGo" (2020)
Silver et al., "Mastering the game of Go without human knowledge" (Nature 2017)
Rich Sutton, "The Bitter Lesson" (2019)
Andy Jones, "Scaling Scaling Laws with Board Games" (2021)