Overview
ARC-AGI-3 (Abstraction and Reasoning Corpus for AGI), designed by François Chollet (creator of Keras) and released in March 2026, delivers a humbling result: humans score ~100% while the most advanced AI systems score below 1% on its unfamiliar interactive tasks. This is not a marginal gap—it is an abyss. The same models that beat human experts on MMLU, win math Olympiad gold, and out-code professionals collapse when placed in a novel environment.
> - Benchmark: ARC-AGI-3 > - Designer: François Chollet > - Released: March 2026 > - Website: https://arcprize.org/ | Paper: arXiv:2603.24621
Key Points
- Crystallized vs. fluid intelligence (Cattell, 1963): LLMs are essentially massive compression of crystallized intelligence—knowledge retrieval and pattern matching from internet-scale training data. Their 90%+ of impressive performance comes from here. Fluid intelligence—reasoning from scratch on never-seen problems—remains far below human level, and it cannot be solved by scaling.
- Three generations of ARC: ARC-AGI-1 (static grids, 2019) resisted pretraining scaling for 5 years—base LLMs stayed under 10% even as model scale grew 50,000x. High scores (o3: 87%, Claude Opus 4.6: 93%, Gemini 3 Deep Think: 96%) came largely from test-time reasoning and probable benchmark contamination. ARC-AGI-2 (2025) added multi-step, context-dependent, symbolic tasks; every AI paradigm dropped 2-3x consistently (e.g., Claude Opus 4.6: 93.0% → 68.8%; resource-constrained winner NVARC: 55.5% → 24.0%), while humans stayed near 100%. ARC-AGI-3 (2026) is an interactive turn-based environment with no instructions and no examples—agents must explore, build internal world models, infer goals, and plan under action budgets.
- Three structural LLM flaws (per the paper and arXiv:2502.07190, Tencent WXG/HKUST): 1. Limited skill composition: fine-tuning on atomic operations improves atoms but not composed tasks—no true compositional reasoning. 2. Encoding barriers: LLMs struggle to translate numeric grids into visual/spatial concepts—a perception-abstraction coupling bottleneck. 3. Left-to-right decoding: next-token prediction is linear, while abstract reasoning is iterative, hypothesis-driven, and backtrackable. Chain-of-Thought only simulates thinking within this linear flow.
- Benchmark contamination: Gemini 3 verbalized color mappings ("Target is Green (3). Pattern is Magenta (6)") never provided in validation prompts—evidence ARC-style tasks leaked into training data. Its high scores may reflect retrieval, not reasoning. Interactive environments counter this: every step is novel and impossible to memorize.
- ARC-AGI-3 paper: arXiv:2603.24621
- ARC-AGI-2 paper: arXiv:2604.02434
- ARC Prize 2025 technical report: arXiv:2601.10904
- ARC-AGI survey: arXiv:2603.13372
- LLM fluid intelligence analysis: arXiv:2502.07190 (Tencent WXG / HKUST)
- Chollet YC interview: https://www.ycombinator.com/library/NP-fran-ois-chollet-arc-agi-3
- ARC Prize: https://arcprize.org/ | ARC Prize 2026: https://arcprize.org/competitions/2026/arc-agi-3
- Cattell, R. B. (1963). Theory of fluid and crystallized intelligence
- Chollet (2019). "On the Measure of Intelligence"
What ARC-AGI-3 Measures
Five core cognitive capabilities: exploration (acting to gather information without intrinsic curiosity), environment modeling (inferring world rules causally, not statistically), goal inference (understanding intent without explicit instructions), planning (efficient strategies under action budgets), and abstraction (extracting general rules from instances). Their intersection is precisely what current AI lacks.
Possible Paths Forward
The most successful ARC solutions are neuro-symbolic hybrids and program synthesis rather than pure neural networks. ARC Prize 2025 winner NVARC used a refinement loop: generate candidate programs, validate on examples, revise from feedback, repeat. ARC Prize 2026 offers a $2M prize pool with a $700K Grand Prize for the first open-source system to reach 100% on ARC-AGI-3—signaling the industry needs new research directions, not bigger models.
Takeaway
Current AI possesses all of humanity's knowledge but lacks basic fluid reasoning. As Chollet argues, test-time reasoning is computation-for-intelligence trade, not genuine abstraction. ARC-AGI-3's value is puncturing a collective illusion: we thought we were building intelligence; we actually built an enormous knowledge retrieval system. Real intelligence is not how much you know—it is how fast you can learn when facing the unknown.