On the ARC-AGI-3 public leaderboard, the combination of Claude Opus 4.8 + Fable 5 scores only 42.83% when orchestrated by Claude Code, but jumps to 98.98% when wrapped in a harness called Schema — an absolute improvement of 56.15 percentage points. Same models, different way of using them. Under the same harness, GPT-5.6 Sol reaches 95.35%. This suggests that the harness has become the dominant abstraction layer of the agent era, while model weights are the secondary variable.
What Happened
Schema's ARC-AGI-3 Public Leaderboard Scores
On July 17, a project called Schema surfaced on Hacker News: an agent harness that encodes the ARC-AGI-3 environment's world model as a runnable program rather than a vector representation. The paper and code are available at schema-harness.github.io.
Its RHAE scores on the 25 games of the ARC-AGI-3 public set:
| Configuration | RHAE Score | |------|----------| | Schema Harness + Claude Opus 4.8 + Fable 5 | 98.98% | | Schema Harness + GPT-5.6 Sol xhigh + Sol max | 95.35% | | Claude Code + Opus 4.8 + Fable 5 (controlled baseline) | 42.83% | | GPT-5.6 Sol max (ARC Prize official best single-variant) | 13.33% |
In the controlled comparison, only the harness changed — the models stayed the same — a 56.15 percentage-point gain.
How Schema Works
Schema's core idea is to have the model encode the environment as a Python program in a world_model.py file, instead of acting on the game directly. Three hard constraints:
1. The world model must be encoded as a runnable step(state, action) program
2. Every recorded transition must pass backtesting validation before use
3. All actions can only be sent via commit_actions; any prediction error immediately discards the remaining plan
The outer loop is four steps: observe → deliberate → execute → record. Within a single deliberation, the inner loop is also four steps: theorize (coding) → certify (backtest) → plan (BFS) → commit. The model first writes code predicting what will happen next, then replays historical data to verify it; only after passing does it run BFS to find the shortest path and commit actions.
If a single prediction is off by even one cell, Schema immediately stops the remaining plan and uses the error as a counterexample to revise the step() program. The entire world model is a text file — diffable and readable — which is the source of its interpretability and the fundamental reason it nearly maxes out all 25 games.
Historical Progress Curve
| Time | RHAE | |------|------| | March 2025 | 0.51% | | July 2025 (GPT-5.6 Sol max) | 7.78% | | GPT-5.6 Sol max / Public | 13.33% | | Claude Code + Opus 4.8 + Fable 5 (baseline) | 42.83% | | Schema + Opus 4.8 + Fable 5 | 98.98% |
From 0.51% to near-perfect within 18 months. Schema represents the largest single jump in that progression.
Deep Analysis
"How You Use the Model Matters More Than the Model Itself" — The Rise of the Harness Abstraction
The core claim holds up: Claude Opus 4.8 and GPT-5.6 Sol are flagship models, widely regarded as among the strongest base models. Under a general-purpose harness (Claude Code), they score 42.83% and 13.33% on ARC-AGI-3. Wrapped in a harness specifically designed for "explicit world model + backtest verification + plan/act separation," both models approach 100%.
This points to a concrete path for the AI application layer: the competitive battleground for frontend engineering, product differentiation, and agent frameworks is no longer base models, but harness orchestration.
OpenAI's Operator, Anthropic's Claude Code, Manus, Genspark, Devin — these products are all, broadly speaking, harnesses. Schema turns this abstraction layer into readable code and a verifiable paper.
Joint Solving of State Attribution + Mechanism Discovery
The most interesting part of the Schema paper is not the score, but that it solves a problem previously attacked separately by VIGA and WorldCoder: knowing simultaneously "what the state is" and "what the rules are."
- State attribution: which pixel patterns correspond to objects, the player, walls, counters.
- Mechanism discovery: how the state changes when a key is pressed.
- The 98.98% is self-reported, not independently verified by ARC Prize. Schema's performance on the semi-private set is unknown.
- The GPT-5.6 Sol row is not a strictly controlled ablation. The 95.35% uses an xhigh + max fallback configuration, while 13.33% is the official best single variant — the model configurations are not fully equivalent.
- Schema's result covers only the 25 public games; stability when generalizing to new environments requires further evaluation.
- An open-source harness abstraction does not equal open-source harness engineering capability. The paper publishes the idea, but embedding it into production-grade agent tools like Claude Code or Cursor is another matter.
- Schema Harness paper and project page: https://schema-harness.github.io/
- Hacker News discussion (July 17)
- ARC-AGI-3 public leaderboard baselines: ARC Prize official public dataset
These two problems cannot be solved independently — a plausible-looking state partition can be falsified by the inability to find consistent transition rules. Schema's solution is to encode both in the same editable program: the model can revise rules while observing, or rehypothesize the state while revising rules.
The paper draws an analogy to the birth of special relativity: the Lorentz route (keep the ether, change the rules) versus the Einstein route (abandon the ether as part of the state — change "what the state is"). Schema encourages the model to take the Einstein route — when failures can't be explained by rules, the model is allowed to directly modify the definition of the state.
If this approach generalizes, it applies beyond ARC-AGI-3: any environment with unknown rules — robotic manipulation, production-line anomaly handling, long-running ops workflows — can adopt the Schema framework.
A Real Case: The LS20 Level
The paper shows a concrete iteration trajectory: Opus 4.8 commits the action sequence [2,2,2,3,2,2] in level LS20, and the Block repaints the indicator maroon — 77 cells predicted incorrectly.
An ordinary agent would retry a different action sequence. Schema has the agent immediately halt the remaining plan, write the error into world_model.py as a counterexample, revise the rule for "which colors represent the indicator," backtest, then run BFS for the shortest path. Seven levels, 642 steps versus a human's 780 — RHAE 100%.
The key design: a single prediction mismatch stops execution, and the mismatched transition becomes a counterexample for revising step(). This closes the trial-and-error-and-revise loop at every granularity of agent decision-making.
Why It Matters
1. The harness abstraction layer is the dominant battleground for AI agents. OpenAI Operator / Anthropic Claude Code / Manus / Devin are all, in essence, expressions of harness design. Schema makes that layer readable, reproducible, and falsifiable. 2. Swapping the harness alone changes the score by 56 points — the hardest data yet pulling industry attention from model iteration speed back to harness design. The next upgrades of tools like Cursor, Claude Code, and Codex will unfold on this abstraction. 3. Schema is fully open source with a public paper — the first near-perfect result in the ARC-AGI-3 series, independently reproducible by anyone with access to frontier model APIs. The harness is no longer a closed black box of OpenAI/Anthropic. 4. The paradigm of "explicit world model + backtest verification + plan/act separation" transfers to other tasks: robotic manipulation, production-line anomaly handling, and long-running ops workflows can all use similar approaches.
Risks and Open Questions
---
Sources: