Player-1: Learning to Play Games from Synthetic Experience Beyond Demonstrations

Forum topic · 小凯 · 2026-10-16

Summary

Player-1 is a post-training framework that teaches computer-use agents to control games through synthetic experience rather than relying solely on demonstrations. The approach extends the computer-use action space with relative pointer motion, sustained inputs, and composite actions. In its supervised initialization stage, the models are fine-tuned on 6,044 curated gameplay trajectories pairing recorded actions with task instructions and visual reasoning. The authors also introduce Player-1-Gym, a suite of sixteen synthetic 2D and 3D environments designed around general game-control needs, featuring programmatically verifiable rewards and randomized observations and control dynamics to support multi-environment PPO reinforcement learning. The resulting models, Player-1-9B and Player-1-35B-A3B, improve success rates on MineDojo-300 by 27.8 and 17.7 percentage points over their SFT policies. Player-1-9B reaches 38.33% success compared to 12.63% for the strongest evaluation baseline, with gains also shown on ViZDoom-10 and GameWorld. Broader synthetic environment coverage improves transfer, and composite keyboard-mouse actions are used increasingly during RL.

Overview

Paper: Player-1: Learning to Play from Synthetic Experience (arXiv 2610.20584)
Field: Computer Vision / Game-Playing Agents
Authors: Mingyu Ouyang, Yiran Wang, Siyuan Hu, Xingyilang Yin, Huiqiang Sun, Guangzhi Wang, Junfu Pu, Cheng Cheng, Honghao Chen, Ying Shan, Mike Zheng Shou, Hwee Tou Ng

Abstract (translated)

Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions. However, games, with their visually rich worlds and diverse mechanics, remain challenging and offer a demanding next step toward general digital control. Current game-agent learning pipelines often rely on large-scale demonstrations with limited coverage of dynamic game states, while scalable environments for interactive training remain difficult to obtain. We introduce Player-1, a post-training framework for learning game control beyond demonstrations through synthetic experience.

The framework extends the computer-use action space with relative pointer motion, sustained inputs, and composite actions. For supervised initialization, computer-use models are fine-tuned on 6,044 curated gameplay trajectories, pairing recorded actions with task instructions and visual reasoning.

The authors also build Player-1-Gym, a collection of sixteen synthetic 2D and 3D environments designed around general game-control requirements. Programmatically verifiable rewards and randomized observations and control dynamics support multi-environment PPO reinforcement learning.

Results

MineDojo-300: Player-1-9B and Player-1-35B-A3B improve success rates by 27.8 and 17.7 percentage points over their SFT policies, respectively.
Player-1-9B achieves 38.33% success versus 12.63% for the strongest evaluation baseline.
Additional gains are demonstrated on ViZDoom-10 and GameWorld.
Broader synthetic environment coverage improves transfer capability, and composite keyboard-mouse actions are used with increasing frequency over the course of RL training.

Conclusion

Learning from synthetic experience substantially improves game-control capabilities beyond supervised initialization, and the gains generalize beyond the RL training suite itself.