Prime Agent: When AI Builds Its Own Tools — The Recursive Harness That Scored 95.5% on ARC-AGI-3

Forum topic · 小凯 · 2026-08-25

Summary

Prime Agent, an open-source self-improving agent harness from Prime Intellect, reportedly lifts performance on the ARC-AGI-3 abstract reasoning benchmark from roughly a 30% LLM baseline to 95.5%. Its core ideas: a persistent IPython REPL so the model can write and rerun code across a session, recursive abstraction (defining reusable helper functions and tools), test-time compute, and a 'Continual Harness' that retains histories, memories, skills, prompts, and subagent specifications across tasks. Recursive subagents communicate directly with each other, specialize by role, and share the same REPL environment. Additional components include an 'Agents View' for human observation and a resource-accounting system to cap compute and API costs. The forum post explains why this works — converting closed-book reasoning into open-book experimentation with code execution, iterative trial-and-error, and problem decomposition — and also offers caveats: the 95.5% may come from brute-force enumeration rather than human-like abstraction, and the architecture brings higher compute cost, latency, complexity, and new security attack surfaces. The author frames Prime Agent as evidence that AI progress may come from smarter scaffolding and cognitive tooling rather than simply larger models. Links: https://arcprize.org/ and https://github.com/PrimeIntellect-ai/prime-agent.

When AI Learns to Build Its Own Tools: The Recursive Revolution of Prime Agent

This post is a Chinese-language analysis of Prime Agent, a self-improving agent harness built on Recursive Language Model (RLM) abstractions. Below is a structured English summary of the key arguments and details.

Key points

Headline result: Prime Agent reportedly achieves 95.5% on ARC-AGI-3, versus a ~30% baseline for state-of-the-art LLMs (GPT-4, Claude 3 class models).
ARC-AGI-3 background: Designed by François Chollet to test fluid intelligence — abstracting rules from examples and applying them to new grid puzzles. It probes compositional, variable-binding, topological, arithmetic, and geometric reasoning.

Why traditional LLMs struggle

The author argues LLMs are trapped by:

Fixed context window limits — only a small slice of knowledge is active at once
A "mental arithmetic only" mode: no code execution, no scratchpad tools, no external verification, no persistence of intermediate results, no delegation

Prime Agent's five core components

1. Persistent Python REPL — a never-closing IPython environment where the model defines and reuses helper functions, following recursive abstraction (layered toolchains where each layer only depends on interfaces). Enables test-time compute: running independent calculations, validating conjectures with scripts, and enumerating possibilities before answering. 2. Continual Harness — retains across tasks: histories, memories, skills (reusable code snippets), validated prompts, and subagent specifications. 3. Recursive subagents — direct agent-to-agent communication without a central dispatcher; specialized roles (code generation, debugging, retrieval, verification) sharing the same persistent REPL. 4. Agents View — a human-observation window for inspecting agent state, progress, and generated code (important for auditing and safety). 5. Resource accounting — tracks per-agent compute, API costs, and task budgets to prevent runaway spending.

Why the architecture works

The post's central framing: Prime Agent turns a closed-book exam into an open-book experiment.

| Capability | Traditional LLM | Prime Agent | |---|---|---| | Verification | Mental math only | Write Python to verify | | Iteration | One-shot answer | Repeated trials and comparison | | Tool reuse | Start from zero | Persist helper functions | | Decomposition | Linear reasoning | Parallel subagents | | Experience | Independent runs | Skills and memories across tasks |

A worked example shows a color-to-number mapping puzzle solved by writing an analysis script, verifying on second examples, applying the mapping, and checking consistency rules.

Reported broader results: matches or exceeds native/leading frameworks on long-context coding, GPU kernel generation, simulator construction, and autonomous nanoGPT speedruns; continuous technical progress on Factorio.

Caveats and reflection

The 95.5% may reflect brute-force enumeration and validation rather than human-like abstract understanding — like solving a Rubik's cube by scripting all move sequences versus knowing group theory.
Costs: higher compute, latency, system complexity, and new security surfaces (arbitrary code execution, privilege escalation in a persistent REPL).
The deeper thesis: progress is shifting from "bigger models" to "smarter scaffolding" — augmenting a fixed model with tools, memory, and collaboration can yield qualitative jumps.

References

Karten, S., Zhang, A. L., Thomas, K., et al. (2026). *Prime Agent: A Self-Improving RLM Harness*. arXiv preprint.
Chollet, F. (2019). *On the Measure of Intelligence*. arXiv:1911.01547.
ARC-AGI-3 Benchmark: https://arcprize.org/
Prime Agent open-source code: https://github.com/PrimeIntellect-ai/prime-agent

> "We shape our tools, and thereafter our tools shape us." — Marshall McLuhan