Reward Size Determines Reinforcement Learning Efficiency: Dopamine Signal Duration Is Key

Forum topic · 小凯 · 2026-07-03

Summary

A Science paper from HHMI's Janelia Research Campus (Gong, Martell, Dudman, Coddington; DOI 10.1126/science.aeb0813) challenges the long-held assumption that learning speed depends mainly on repetition count rather than reward magnitude. Studying naive mice across five behavioral paradigms, the authors found that large rewards dramatically accelerate initial learning: thirsty mice mastered a task in a single day with a few large rewards (100 microliters), whereas thousands of small rewards (5 microliters) required weeks. Large rewards also eliminated individual variability, bringing all subjects to expert performance within days, and separately boosted within-session learning, cross-session retention, and task engagement—with engagement being the largest factor. Ventral striatal dopamine release duration and amplitude scaled with reward size, and optogenetically prolonging dopamine reward responses reproduced most, but not all, of the learning benefits of large rewards. The findings imply that reinforcement learning efficiency in animals has been traditionally underestimated. For AI, the results suggest that sparse or minimal reward schemes in RL experiments may substantially underestimate a system's learning potential; larger reward signals imply faster learning rates and convergence.

Paper Overview

Field: Neuroscience / Reinforcement Learning / Dopamine Authors: Sheng Gong, Alyssa Martell, Joshua T. Dudman, Luke T. Coddington Institution: HHMI Janelia Research Campus Published: 2026-05-21 Journal: Science (Volume 392, Issue 6800) DOI: 10.1126/science.aeb0813

Abstract

Standard animal learning studies minimize individual rewards to maximize the number of reinforced behavior repetitions. This work examined how reward size affects initial learning in naive mice across five behavioral paradigms. Large rewards significantly improve learning efficiency through dissociable effects on within-session learning, across-session learning, and task engagement. The duration and amplitude of ventral striatal dopamine release scale with reward size, and prolonged optogenetic enhancement of dopamine reward responses reproduced most—but not all—of the learning benefits conferred by oversized rewards. These findings suggest that animals' reinforcement learning efficiency has been traditionally underestimated, and that dopamine signaling of reward modulates task engagement in proportion to absolute reward magnitude.

Key Findings

Challenging a traditional assumption: Decades of neuroscience assumed learning speed depended mainly on the amount of experience (repetitions), independent of reward size.
The "cookie vs. M&M" effect: Thirsty mice mastered a task within a single day using a few large rewards (100 microliters), while thousands of small rewards (5 microliters) took weeks.
Eliminating individual differences: With large rewards, all subjects reached expert level within days; with small-reward protocols, learning speed varied widely (from 1 week to 1 month).
Dopamine mechanism: Large rewards produce not only larger dopamine peaks but—more critically—prolong the duration of the dopamine signal.
Three learning components: Improved per-repetition learning efficiency, stronger cross-day memory retention, and increased task engagement—with engagement being the largest determinant.
Optogenetic validation: Artificially prolonging the dopamine signal associated with small rewards successfully reproduced most of the learning benefits of large rewards.

Implications for AI

This paper has deep implications for reinforcement learning in AI: the sparse or minimal reward schemes commonly used in current RL experiments may severely underestimate a system's learning potential. Larger reward signals (longer dopamine waves) imply higher learning rates and faster convergence.

--- *Auto-collected on 2026-07-03*