Comparative Reversal Learning Reveals Rigid Adaptation in LLMs Under Non-Stationary Uncertainty

Forum topic · 小凯 · 2026-04-07

Summary

This paper evaluates how large language models adapt in non-stationary environments using a two-option probabilistic reversal-learning task with three latent states, comparing deterministic and stochastic transition schedules. The study evaluates DeepSeek-V3.2, Gemini-3, and GPT-5.2 against human behavioural data. Across all models, win-stay behaviour was near ceiling while lose-shift was markedly attenuated, revealing an asymmetric use of positive versus negative evidence. DeepSeek-V3.2 showed extreme perseveration after reversals and weak acquisition, while Gemini-3 and GPT-5.2 adapted faster but remained less loss-sensitive than humans. Notably, random transitions amplified reversal-specific persistence without uniformly reducing total wins, showing that high aggregate payoff can coexist with rigid adaptation. Hierarchical reinforcement-learning fits suggest rigidity stems from weak loss learning, inflated policy determinism, or value polarisation via counterfactual suppression. The authors propose reversal-sensitive diagnostics and volatility-aware evaluation methods for LLMs.

Paper Overview

Field: Machine Learning
Authors: Haomiaomiao Wang, Tomas E Ward, Lili Zhang

Abstract (translated)

Non-stationary environments require agents to revise previously learned action values when contingencies change. We treat large language models (LLMs) as sequential decision policies in a two-option probabilistic reversal-learning task with three latent states and switch events triggered by either a performance criterion or timeout. We compare a deterministic fixed transition cycle to a stochastic random schedule that increases volatility, and evaluate DeepSeek-V3.2, Gemini-3, and GPT-5.2, with human data as a behavioural reference.

Across models, win-stay was near ceiling while lose-shift was markedly attenuated, revealing asymmetric use of positive versus negative evidence. DeepSeek-V3.2 showed extreme perseveration after reversals and weak acquisition, whereas Gemini-3 and GPT-5.2 adapted more rapidly but still remained less loss-sensitive than humans. Random transitions amplified reversal-specific persistence across LLMs yet did not uniformly reduce total wins, demonstrating that high aggregate payoff can coexist with rigid adaptation. Hierarchical reinforcement-learning (RL) fits indicate dissociable mechanisms: rigidity can arise from weak loss learning, inflated policy determinism, or value polarisation via counterfactual suppression. These results motivate reversal-sensitive diagnostics and volatility-aware models for evaluating LLMs under non-stationary uncertainty.

Key Findings

All LLMs showed a "win-stay strong, lose-shift weak" asymmetry — near-ceiling use of positive evidence but attenuated use of negative evidence.
DeepSeek-V3.2 exhibited extreme perseveration after reversals and weak acquisition; Gemini-3 and GPT-5.2 adapted faster but were still less loss-sensitive than humans.
Stochastic transitions increased reversal-specific persistence without uniformly lowering total wins — high aggregate payoff can coexist with rigid adaptation.
Hierarchical RL modelling identified dissociable sources of rigidity: weak loss learning, over-confident policy determinism, and value polarisation via counterfactual suppression.