Semifactual Credit-Augmented Policy Optimization (SCAPO): A Causally-Inspired GRPO Variant for LLM Reasoning

Paper Overview

Field: NLP Authors: Junshu Pan, Zhizhang Fu, Shulin Huang, et al. Published: 2026-09-30 arXiv: 2609.26735

Summary

Reinforcement learning with verifiable rewards (RLVR) has improved the reasoning capabilities of large language models (LLMs), yet their predictions remain sensitive to task-irrelevant prompt features. The authors investigate this sensitivity through semifactual prompt interventions — perturbations that preserve the underlying problem and its answer.

Key Findings

• The analysis reveals substantial variation in token-level sensitivity to semifactual interventions.
• Suppressing high-drift token candidates during decoding improves reasoning accuracy without updating model weights.
• These findings highlight a limitation of Group Relative Policy Optimization (GRPO): it assigns the same outcome-derived advantage to every response token, potentially reinforcing spurious dependence alongside useful reasoning.

Proposed Method: SCAPO

Motivated by these observations, the paper introduces Semifactual Credit-Augmented Policy Optimization (SCAPO), a causally-inspired GRPO variant that incorporates semifactual stability into token-level credit assignment:

• SCAPO measures token probability drift of a fixed response under semifactual interventions.
• It uses a normalized stability score to down-weight the advantage of relatively unstable tokens early in training.
• Stability alone does not grant additional credit.

Results

• On Qwen3-4B-Base and Qwen3-1.7B-Base, SCAPO improves AIME 2024-2026 accuracy by +5.63 and +4.17 percentage points over GRPO, respectively.
• At both model scales, SCAPO achieves the best results among compared methods on most math benchmarks and all out-of-distribution benchmarks.

Original Abstract

> Reinforcement learning with verifiable rewards (RLVR) has improved the reasoning capabilities of large language models (LLMs), yet their predictions remain sensitive to task-irrelevant prompt features. We investigate this sensitivity through semifactual prompt interventions that preserve the underlying problem and its answer. Our analysis reveals substantial variation in token-level sensitivity and shows that suppressing high-drift token candidates during decoding improves reasoning accuracy without updating model weights. These findings highlight a limitation of Group Relative Policy Optimization (GRPO), which assigns the same outcome-derived advantage to every response token and may reinforce potential spurious dependence alongside useful reasoning.

--- *Auto-collected on 2026-10-02*