FADE: Factored Ability-Difficulty Filtering for Online RLVR Prompt Scheduling

Forum topic · 小凯 · 2026-11-03

Summary

FADE (Factored Ability-Difficulty Estimation) is a new prompt scheduling method for group-based reinforcement learning with verifiable rewards (RLVR). RLVR can only learn from prompts where sampled completions produce mixed results; groups that all pass or all fail yield zero relative advantage and waste generation budget. Existing adaptive schedulers track per-prompt pass rates, but this entangles two quantities that evolve at different speeds: the policy's changing competence across reasoning skills and each prompt's intrinsic difficulty. FADE is a bilinear state-space filter that separates these two timescales. It models each prompt's success logit as an inner product between a whitened K-dimensional skill loading and a fast-drifting policy ability vector, minus a slowly shrinking scalar item bias. Using Pólya-Gamma augmentation, binary outcomes in each batch update the K-dimensional ability posterior in closed form via a rank-K Woodbury step, instantly propagating ability changes to all unobserved prompts sharing those skills while item biases remain stable. A closed-form utility function, derived from posterior mean and variance, estimates the probability of sampling a non-degenerate group. Across five reasoning benchmarks, two base models, and four scheduling paradigms under equal rollout budgets, FADE consistently improves downstream accuracy and calibration with under 0.6 ms of per-step overhead. Paper: arXiv:2611.03625.

Paper Overview

Field: Machine Learning Authors: Tomas Duncun, Jonah Ellison, Mateo Ruiz, Hanae Fujita, Ingrid Solberg Published: 2026-11-02 arXiv: 2611.03625

Abstract

Group-based reinforcement learning with verifiable rewards (RLVR) learns only from prompts whose sampled completions disagree: all-pass and all-fail groups produce zero relative advantage and burn generation budget without moving the policy. Adaptive schedulers try to steer rollouts toward informative prompts, yet they rely on per-prompt pass-rate trackers that entangle two quantities evolving at very different speeds—the policy's shifting competence across reasoning skills and each prompt's intrinsic difficulty. When a single exponential average or Beta counter tracks both at once, short memory scrambles the relative ordering of prompts whereas long memory reports stale pass rates.

The authors introduce FADE (Factored Ability–Difficulty Estimation), a bilinear state-space filter that separates the two timescales:

Each prompt's success logit is modeled as an inner product between a whitened K-dimensional skill loading and a fast-drifting policy ability vector, minus a slowly shrinking scalar item bias.
Using Pólya-Gamma augmentation, each batch of binary outcomes updates the K-dimensional ability posterior in closed form via a rank-K Woodbury step, instantly propagating ability changes to all unobserved prompts sharing those skills, while item biases remain stable.
From the posterior mean and variance, FADE derives a closed-form utility function: the probability of sampling a non-degenerate group.

Results

Across five reasoning benchmarks, two base models, and four scheduling paradigms, under equal rollout budgets, FADE consistently improves downstream accuracy and calibration, with less than 0.6 ms of additional overhead per step.

--- *Auto-collected on 2026-11-04*