VISTA: MIT's Kaiming He Group Gives Models a Visual Memory Album for ARC-AGI-3, Achieving 25/25

Forum topic · 小凯 · 2026-10-18

Summary

ARC-AGI-3 is a benchmark of interactive grid-world mini-games (64x64), typically solved by converting each frame into text token grids. VISTA, from MIT's Kaiming He group (arXiv 2610.02200), takes a different approach: the model views rendered images directly, maintains a lossless archive of every frame encountered, and can zoom into archived regions during reasoning—e.g., zooming into the hotbar to distinguish iron ore from stone in a Minecraft clone, or replaying the first frame to see a maze larger than one screen. With VISTA, Claude Opus 5.0 cleared all 25 public games, using 57.4% fewer actions than first-time human players (efficiency score 40.68 to 100.00); GPT-5.6 Sol achieved 99.00 efficiency. The surprising result is cost: a 64x64 text grid costs ~4,000 tokens per frame, while a 512x512 rendered image costs only ~308 image tokens—13x cheaper while preserving spatial structure. Across a full game, text used 71.9M tokens versus 30.7M for images. The paper claims this is the first vision-based system to reach near-perfect scores on ARC-AGI-3 without text transcription.

VISTA: Models Don't Lack Brains, They Lack Photo Albums

ARC-AGI-3 is a set of interactive mini-games: a 64x64 grid world where players must figure out the rules and find a path to victory. After this benchmark was released, the mainstream approach was to compress every frame into text—turning the grid into rows of color codes and feeding it to language models as a text puzzle.

MIT's Kaiming He group took a different approach: let the model look at the images directly. The paper is called VISTA, arXiv number 2610.02200, and two of its five authors (Linlu Qiu, Keya Hu) are also authors of the ARC-AGI-3 paper itself—the people who wrote the exam came back to build the equipment.

The Three-Piece Kit

VISTA gives the model three things.

First, it views rendered images directly, no text conversion. Second, a lossless photo album: every frame encountered during interaction is archived as-is—nothing compressed, nothing discarded. Third, a magnifying glass: during reasoning, the model decides which archived frames to revisit and which regions to zoom in on. Playing a Minecraft clone, it zooms into the hotbar to distinguish iron ore from stone; playing Pac-Man, it flips back to the first frame to see the full maze—the maze is bigger than the screen, so you can't memorize it in one glance; looking back is the only way. It also keeps a text notebook alongside, updated as it plays.

The control group uses the standard approach: a 175K context window, where the earliest turns are dropped once it fills up—the agent has no long-term memory and forgets the beginning by the end. VISTA's album lives outside the context window: raw frames take up no room, retrieved only when needed.

Perfect Score

With VISTA attached, Claude Opus 5.0 cleared all 25 public games, using 57.4% fewer actions than first-time human players, with the efficiency score raised from 40.68 to a full 100.00. Swapping in GPT-5.6 Sol: all 25 games cleared, efficiency 99.00. The paper states this is the first visual system to achieve full or near-full scores on this benchmark without text transcription.

It generalizes on three other benchmarks as well: AI GameStore success rate 99.04%, GameWorld 63.33% (human baseline: two CS graduate students), BabyVision 63.23%.

The Most Surprising Number

The most counterintuitive number in the paper isn't on the scorecard—it's on the bill.

A 64x64 text grid costs about 4,000 tokens. The same frame rendered as a 512x512 image costs about 308 image tokens. Converting an image to text preserves less structure and costs 13 times more. Over a full game, the text version used 71.9 million tokens; the image version, 30.7 million.

The old saying "a picture is worth a thousand words" gets a precise meaning here: one image beats its own text description by 13x. For years, people converted games into text and fed them to models—losing spatial structure while paying more in tokens.

Footnote

The code repository listed in the paper currently returns a 404; an earlier version appeared as a blog post in August. The core model hasn't changed—what changed is its treatment: a visible interface, and an album you can flip through. The perfect score comes not from a smarter brain, but from finally completing the eyes and the memory.

308 versus 4,000—images are cheaper than text. That's worth remembering forever.