[论文] Comparative reversal learning reveals rigid adaptation in LLMs under non-stationary uncertainty

论文概要

研究领域: ML 作者: Haomiaomiao Wang, Tomás E Ward, Lili Zhang 发布时间: 2025-04 arXiv: 2503.xxx4

中文摘要

非平稳环境要求智能体在情境变化时修正先前学习的动作价值。我们将大语言模型（LLMs）视为序列决策策略，在一个具有三个潜在状态的双选项概率反转学习任务中，转换事件由性能标准或超时触发。我们将确定性固定转换周期与增加波动性的随机随机时间表进行比较，评估了DeepSeek-V3.2、Gemini-3和GPT-5.2，以人类数据作为行为参考。跨模型来看，赢-保持在接近上限水平，而输-转换明显减弱，揭示了对正负面证据的不对称使用。DeepSeek-V3.2在反转后表现出极端的固执和弱习得，而Gemini-3和GPT-5.2适应更快，但仍不如人类对损失敏感。随机转换放大了LLMs中反转特定的固执，但并未统一减少总胜利次数，表明高总体回报可以与刚性适应共存。分层强化学习（RL）拟合表明存在可分离的机制：刚性可能源于弱损失学习、膨胀的策略确定性，或通过反事实抑制的价值极化。这些结果推动了在非平稳不确定性下评估LLMs的反转敏感诊断和波动感知模型的发展。

原文摘要

Non-stationary environments require agents to revise previously learned action values when contingencies change. We treat large language models (LLMs) as sequential decision policies in a two-option probabilistic reversal-learning task with three latent states and switch events triggered by either a performance criterion or timeout. We compare a deterministic fixed transition cycle to a stochastic random schedule that increases volatility, and evaluate DeepSeek-V3.2, Gemini-3, and GPT-5.2, with human data as a behavioural reference. Across models, win-stay was near ceiling while lose-shift was markedly attenuated, revealing asymmetric use of positive versus negative evidence. DeepSeek-V3.2 showed extreme perseveration after reversals and weak acquisition, whereas Gemini-3 and GPT-5.2 adapt...

自动采集于 2026-04-07

#论文 #arXiv #ML #小凯 #自动采集