English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Anthropic Drone-Bench: Fable 5 Becomes First AI Model to Autonomously Pilot a Drone, But Cross-Room Navigation Still Fails

Forum topic · 小凯 · 2026-07-25

Summary

Anthropic and Andon Labs have released Drone-Bench, a benchmark testing AI models on indoor quadcopter drone control. The benchmark splits the task into five subtasks: Reconstruct (converting office video into 3D models and 2D obstacle maps), Localize, Navigate, Detect, and Follow. Testing 15 models from GPT-4o to GPT-5.6 Sol, the results show newer models progressing further, with most stalled at Reconstruct. Fable 5 is the only model to exceed the expert baseline on four of five subtasks, and on a real DJI Tello EDU drone ($129 retail), it clearly outperformed baseline on detect and follow tasks. However, reconstruction errors propagate into localization and navigation, causing cross-room navigation to fail—Fable 5 confidently flew into a wall it mistook for a door. Key observations include Fable 5's local verification behavior (inferring camera tilt from floor-tile vanishing points within 4 degrees) and a reliability gap: the model's best-of-runs performance exceeds its average performance, with researchers estimating the capability frontier leads consistency by about six months. This is Anthropic's third physical-AI project after Project Vend and Project Fetch.

Anthropic and Andon Labs have jointly released a new benchmark called Drone-Bench. The task is specific: have AI models pilot a quadcopter drone in an indoor office environment to locate and track a designated person—essentially a sequel to Project Fetch (the robot dog fetching a ball).

Benchmark Structure

The task is split into five subtasks—a key design choice to avoid a single overall failure wiping out partial progress:

  • Reconstruct: Convert office video into a 3D model, then extract a 2D obstacle map
  • Localize: Match the drone's current position onto the 2D map using office frames with known poses
  • Navigate: Plan a path on the map and fly it, repeatedly calling Localize to correct course
  • Detect: Find a person in a room using a reference photo, outputting a bounding box
  • Follow: Use the bounding box to keep the person centered in the frame

Results

Andon tested 15 models, from GPT-4o to GPT-5.6 Sol. The conclusion: newer models get further through the subtasks, and everyone currently stalls at Reconstruct.

Fable 5 is the only model to cross the baseline—passing all subtasks except Reconstruct. Run end-to-end on a real drone, Fable 5's detection and following clearly beat the baseline. But because Reconstruct errors propagate into Localize and Navigate, cross-room navigation still fails. The article includes a memorable clip: Fable 5 confidently flies toward what it believes is a door—and hits a wall.

Two Details Worth Noting

First, Fable 5 performs local verification in some submissions. In one submission, it inferred the drone camera's tilt angle by analyzing the vanishing points of floor tiles, accurate to within 4 degrees. In another, it first drew a 2D top-down reconstruction of the Follow-task environment, tested its implementation locally and iterated, and only then submitted. These are behaviors of a model doing engineering, not shouting answers.

Second, consistency is an underrated problem. Fable 5 crosses the baseline in at least 4/5 of simulation runs at best—but only 3/5 on average. The researchers explicitly distinguish "one-off success vs. consistent success" and attach a number: the capability frontier leads consistency by roughly six months. In other words, "best-of-runs" performance in early 2026 ≈ Fable 5's average performance today.

Hardware Matters

The drone used in the experiments is a DJI Tello EDU, retailing at $129. The hardware barrier for "AI autonomously controlling a drone" has dropped to consumer-hobbyist level—not industrial equipment costing tens of thousands of dollars.

Stated Limitations

1. Test scenarios use a single indoor office floor plan and a single target person—no outdoor scenes or dense crowds 2. The drone flies slowly 3. The baseline represents what an "AI expert + modern tools" can achieve—not the ceiling of an unassisted human, let alone centralized human-AI collaboration

A Quote Worth Reading

> "We note that in the early days of agentic coding, humans approved nearly every tool call; months later, models were trusted to run long-horizon tasks with minimal intervention. Once models cross the threshold of capability and reliability—like the human-AI team baseline used here—'human oversight' will be seen as a cost rather than a safety net. This is precisely why these decisions must be made carefully, especially in domains involving physical safety and privacy, where efficiency alone should not be the dominant consideration."

The Bigger Picture

This is Anthropic's third consecutive move on the "physical AI" front (Project Vend, Project Fetch, Drone-Bench), all premised on "everyday hardware + modern AI." It's no longer talking about sci-fi narratives like humanoid robots—it has pushed the barrier down to a $129 drone. This means the embodied AI competition entering 2026 H2 is starting to generate empirical evidence on "off-the-shelf hardware + frontier model" combinations.

Original article

Tags

#anthropic#drone-bench#ai-benchmarks#embodied-ai#fable-5#autonomous-drones#andon-labs#physical-ai

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178447091