Firing the Model Off the Robot: NTU's Code-Only-as-Policy Hits 70% on RoboDojo Without Any Runtime Model Calls

Forum topic · 小凯 · 2026-10-23

Summary

A technical report from Nanyang Technological University and Ropedia (arXiv 2610.12369) introduces COAP (Code-Only-as-Policy), a robot control framework that eliminates all model calls at runtime. Instead of querying a VLA or VLM every step, COAP represents the embodied world as explicit state variables across five categories—robot, object, environment, relation, and task—measured from three RGB cameras and proprioception only, and makes every decision in plain code that runs on CPU. An offline loop lets coding agents improve the codebase by tracing failures back to variables, with changes validated by CPU prechecks and merged only if they do not hurt other tasks. On 42 bimanual RoboDojo tasks, COAP achieves 70.24% success versus 31.38% for the leaderboard leader, ranking first on all five dimensions including Memory (89.9%) and Precision (75.1%). The report positions codebases as a medium for recursive self-improvement, though results remain simulation-only.

Firing the Model Off the Robot

Today's mainstream robot policies share a common habit: they query a large model at every single step. VLA models map observations to actions, and Agent Harnesses simply call a VLM once per step for decisions. It's expensive, slow, and the robot's state ends up scattered across model weights or text context—when something goes wrong, it's hard to know where to even start looking. A technical report from Nanyang Technological University and Ropedia (arXiv 2610.12369, posted October 8) takes a different path, starting from an old metaphor: the embodied world is a Turing machine—the tape is the state of the robot and environment, the rules are the policy. As long as the state can be accurately represented, decisions can be written entirely in code, without calling a model even once.

The Three-Layer Approach

The method has three layers. The first is measurement: the runtime code consumes only three color cameras (one on the head, one on each wrist) plus proprioception—joint angles, end-effector pose, gripper command echoes—and a single language instruction. No depth camera, no segmentation, no model services. Measured quantities are stored as explicit variables in five categories: robot, object, environment, relation, and task. Each measurement record carries error, source, and timestamp; objects occluded by the arm retain the last measured pose, and millimeter-precision tasks trigger a close-up re-measurement with the wrist camera.

The second layer is decision-making: all judgments are written over these five categories of state variables, and runs purely on CPU at test time.

The third layer is improvement—where the report's real ambition lies. A coding agent improves the codebase in an offline closed loop: it reads failure scenes, traces back from the failure to the variable that caused it, and modifies the code logic that derived it. Changes are evaluated in parallel as branches under identical conditions, and before simulation they pass two CPU prechecks (step-by-step verification of numerical scenarios plus an inverse kinematics check). Only changes that bring improvements without damaging other tasks' scores get merged.

Results

The results were obtained on 42 bimanual tasks of the RoboDojo benchmark, using the official evaluation client and grading scripts:

COAP (Code-Only-as-Policy): 70.24% overall success rate, 38.9 percentage points above the leaderboard's current best of 31.38%
Strongest world action model: 29.64%
Strongest VLA: 27.96%—π0.5 is on this list of opponents

COAP ranks first on all five dimensions of the benchmark. The largest leads come precisely on Memory (89.9%, ahead by 39.6 points) and Precision (75.1%, ahead by 41.6 points)—the two dimensions that depend most on state retention and measurement accuracy. The benefits of explicit state translated directly into scores.

The Real Pain Point: Cumulative Improvement

The design's real target is cumulative improvement. VLAs need retraining to learn new tasks, and old tasks break: in an experiment cited by the paper, π0.5's average score dropped from 86.9 to 31.4 after sequential fine-tuning on five real tasks. COAP's codebase is the opposite—each merged fix preserves old logic, so capability accumulates monotonically with tasks. The report thus positions COAP as a suitable medium for recursive self-improvement: every change is explicit, measurable, and rollback-capable, and the improvement process is the very process software engineers know. It also reserves a downstream role for itself: successful trajectories produced by this model-free runtime can serve directly as training data for VLAs and Agent Harnesses—the codebase doubles as a data engine.

Limitations

The boundaries must be stated clearly:

All results come from simulation. COAP's perception relies on the benchmark's public camera parameters, robot meshes, and object dimensions—on real hardware these all need re-measuring, and contact parameters like gripper width and transport speed must be retuned.
Offline development is slow—every iteration requires running tests.
Failure attribution shows problems concentrated in state measurement and code logic. The authors put it bluntly: COAP's ceiling depends on how accurate the state representation is and how robust the code logic is.

Perspective

Seen in context, CodeHack previously collapsed action execution into 78 Python skills; this time COAP collapses decision control into code as well—the runtime doesn't retain a single model call. But the phrase "firing the model" isn't quite right: the model hasn't disappeared, it has been reassigned. It's no longer called at every step; it has retreated to the development stage, working when writing code, reading failures, and fixing logic. Verification costs have likewise been relocated—from every step at runtime to every merge in offline testing. The more robots resemble software, the more software engineering's old rules apply: change a line, run a test, and don't break what already works.