Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learning (VRRL)

Summary

This forum post introduces VRRL, a reinforcement learning training framework by Liyan Tang, Fangcong Yin, and Greg Durrett that elicits visually grounded self-reflection in vision-language models. The framework comprises two components: (1) random masking of trajectory prefixes, which encourages the model to recover from incorrect predictions by re-reasoning from partial trajectories, and (2) buffered roll-ins, which expose the model to diverse failure states during training. By training on these deliberately perturbed and failure-rich trajectories, vision-language models learn to detect their own errors, look back at the visual input, and revise predictions in a grounded manner. The paper is available on arXiv as 2607.02490 (cs.CL, cs.CV), posted July 2, 2026. The post was automatically collected on zhichai.net on August 28, 2026. It is a brief index-style entry linking to the arXiv abstract rather than a detailed discussion thread.