ContractRL: Shielded Group-Relative Policy Optimization for Auditable JSON Repair

Forum topic · 小凯 · 2026-10-05

Summary

ContractRL (arXiv 2610.00328) is a contract-constrained sequential repair protocol that treats verifier-guided JSON repair of structured tool calls as a bounded decision process. Instead of regenerating a complete object when only a few fields violate a schema, the policy at each step observes the candidate object, typed verifier feedback, JSON Pointer, immutable repair history, and remaining budget. A contract-derived action mask filters malformed or prohibited RFC-6902 operations before a deterministic validator performs the transition, and a contract-constrained group-relative objective governs patch, retry, and abstention decisions. Experiments across five seeds (192 cases per seed) show ContractRL achieves 0.9362 semantic success rate with 34.4 generated tokens, compared to Patch-SFT (0.9076/44.9) and full regeneration (0.9148/137.2). Policy optimization improves the supervised variant from 0.9186 to 0.9375, and a three-seed paired evaluation shows a +0.0396 semantic gain over Patch-SFT (95% CI [+0.0137, +0.0662], p=0.0039). Feedback, action-mask, budget, and schema-drift analyses link gains to localized corrections, while adversarial and multi-turn evaluations characterize remaining failure modes.

论文概要

领域: NLP 作者: Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Yina Sa, Daren Zha, Jun Xiao 发布时间: 2026-10-05 arXiv: 2610.00328

Introduction

Structured tool calls often fail when only a small number of fields violate a schema or an execution contract. Regenerating the complete object enlarges the action surface and makes repeated repair difficult to audit. ContractRL addresses this by modeling verifier-guided JSON repair as a bounded decision process.

Method

At each step, the policy observes: the candidate object, typed verifier feedback, JSON Pointer, an immutable repair history, and the remaining budget.
A contract-derived action mask filters malformed or prohibited RFC-6902 operations before a deterministic validator performs the transition.
A contract-constrained group-relative objective specifies patch, retry, and abstention decisions, while canonical targets and semantic labels are kept outside the online state until trajectory freeze (shielded group-relative policy optimization).

Results

Under identical verifier information (192 cases per seed, five seeds):

| Method | Semantic success | Generated tokens | |---|---|---| | ContractRL | 0.9362 | 34.4 | | Patch-SFT | 0.9076 | 44.9 | | Full regeneration | 0.9148 | 137.2 |

Policy optimization lifts the supervised variant from 0.9186 to 0.9375.
Independent three-seed paired evaluation vs Patch-SFT: +0.0396 semantic difference (95% CI [+0.0137, +0.0662], p=0.0039).
Feedback, action-mask, budget, and schema-drift analyses attribute gains to localized corrections; adversarial and multi-turn evaluations characterize remaining failure modes.