English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Reframe: Observation-Aligned Multi-Teacher On-Policy Distillation for Visual Agents

Forum topic · 小凯 · 2026-11-10

Summary

Reframe is a post-training framework for visual agents introduced by Haozhi Yuan, Handong Li, Yifan Zhang, and Qianshan Wei (arXiv:2611.12886). It addresses two interrelated challenges in tool use: models may ignore useful tool-returned observations, and poorly grounded actions limit the evidence available for distillation. Unlike standard approaches that align only output distributions, Reframe treats the observation state itself as an alignment object. It operates in two stages: (1) GD-GRPO, which trains spatial and temporal experts with evidence-aware rewards and stochastic removal of raw visual inputs, encouraging grounded evidence and reliance on returned observations; and (2) OG-MOPD, which replaces defective tool actions with canonical ones mid-rollout and resumes distillation from repaired observations. The stages are mutually reinforcing, and neither requires task-specific SFT cold starts. Experiments across 14 image and video benchmarks show Reframe outperforms Mix-RL and original MOPD, with ablations confirming improved observation reliance and tool gains over outcome-based GRPO.

Reframe: Observation-Aligned Multi-Teacher On-Policy Distillation for Visual Agents

  • Field: Computer Vision (CV)
  • Authors: Haozhi Yuan, Handong Li, Yifan Zhang, Qianshan Wei
  • Published: 2026-11-09
  • arXiv: 2611.12886
  • Overview

    Agentic reinforcement learning (RL) and multi-teacher on-policy distillation (MOPD) offer complementary routes to post-training. For visual agents, however, tool use presents two related challenges. Models may overlook useful tool-returned observations, while poorly grounded actions can limit the evidence available for subsequent distillation.

    Reframe is a post-training framework that treats the observation state, rather than only the output distribution, as an object of alignment.

    Key Points

  • Stage 1 — GD-GRPO: Trains spatial and temporal experts using evidence-aware rewards combined with stochastic removal of raw visual inputs, pushing the model to acquire grounded evidence and actually leverage tool-returned observations.
  • Stage 2 — OG-MOPD: Replaces defective tool actions with canonical ones midway through rollouts, then resumes distillation from the repaired observations.
  • Synergy: The two stages are complementary — learning to rely on tool-returned observations makes the policy better positioned to benefit from observation repair.
  • No cold start: Both stages eliminate the need for task-specific SFT cold starts.
  • Results

  • Extensive experiments on 14 image and video benchmarks show Reframe outperforms both Mix-RL and the original MOPD.
  • Ablation studies indicate GD-GRPO significantly improves observation reliance and tool gains compared with outcome-based GRPO.
  • Links

  • Paper: https://arxiv.org/abs/2611.12886
*Auto-collected on 2026-11-11.*

Tags

#computer-vision#reinforcement-learning#knowledge-distillation#visual-agents#post-training#tool-use#arxiv#paper

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637009