Summary
VeriFine is an agent harness framework that scales verification through the co-evolution of a policy, training curriculum, and judge for self-improving embodied AI systems. As self-improving policies expose new failure patterns, fixed judges constrain both optimization feedback and the discovery of useful training examples. VeriFine addresses this with two interleaved loops: a Policy Improvement Loop where a rubric judge diagnoses recurring failures, constructs adaptive curricula, and optimizes the policy; and a Judge Improvement Loop that selectively queries human guidance on informative failure cases and refines the judge through interactive calibration, where humans and the agent resolve disagreements and converge on a target rubric for physical reasoning. Experiments on driving and robot navigation tasks show sustained self-improvement of both policy and judge capabilities across reinforcement learning and supervised fine-tuning. Authored by Zewei Zhou, Rachel Luo, Marco Pavone, Wenhao Ding, and colleagues, the paper is available as arXiv 2610.08761.
Paper Overview
Field: Machine Learning
Authors: Zewei Zhou, Rachel Luo, Yulong Cao, Chaowei Xiao, Chensheng Peng, Boyi Li, Thomas Tian, Zheng Lian, Yan Wang, Jiaqi Ma, Boris Ivanovic, Marco Pavone, Wenhao Ding
Published: 2026-10-06
arXiv: 2610.08761
Abstract
Self-improving policies continually expose new failure patterns, changing what their judges must be able to verify. However, current fixed judges constrain both optimization feedback and the discovery of useful training examples, limiting further self-improvement. This challenge is even more acute in embodied reasoning, where reliable evaluation must account for spatial grounding, causal reasoning, and safety-aware decision-making.
We introduce VeriFine, an agent harness framework that scales verification through the co-evolution of the policy, training curriculum, and judge.
- Policy Improvement Loop: uses a rubric judge to diagnose recurring failures, construct an adaptive curriculum, and optimize the policy.
- Judge Improvement Loop: when progress plateaus and verification becomes a bottleneck, selectively queries human guidance on informative failure cases and refines the judge through interactive calibration — humans and the agent resolve disagreements and converge on a target rubric for physical reasoning. The revised judge then guides the next phase of data selection and policy optimization.
Experiments on driving and robot navigation tasks demonstrate sustained self-improvement of both policy and judge capabilities across reinforcement learning and supervised fine-tuning.
---
*Auto-collected on 2026-10-08*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178635725