Research area: Machine Learning Author: Saad Mankarious Published: 2026-05-26 arXiv: 2505.21639
Abstract
We introduce *Quantum Frog*, a two-player cooperative game built on a novel *quantized-time* mechanic in which the environment advances only when a player acts. Inspired by the classic arcade game Frogger, Quantum Frog requires two frogs to cross an 8×8 grid of traffic and reach the far side together. We use reinforcement learning (RL) as an analytical lens to answer four design questions: (1) how does game difficulty scale with traffic density, (2) what is the optimal single-agent policy and why, (3) how large is the cooperation gap between independent and cooperative two-agent play, and (4) what joint strategy emerges when agents are incentivised to cooperate? We train agents through five escalating stages—Tabular Q-Learning, Deep Q-Network (DQN), Independent DQN (IDQN), and Multi-Agent Proximal Policy Optimisation (MAPPO with a centralised critic), evaluating each against traffic densities of one to six cars.
Key Findings
- Rush strategy is universally optimal: the quantized-time mechanic makes moving directly upward at every step optimal, as time exposure to traffic is minimised.
- Coordination is harder than density: adding an uncoordinated second player is harder than sextupling the traffic for a single expert player.
- Cooperative training pays off: cooperative training recovers +32–34 percentage points of joint success rate relative to independent agents and reduces episode length from ~90 to ~6 steps.
- Emergent strategy is synchronised rushing: the cooperative solution is synchronised rushing, not complex positional coordination, illustrating that shared incentives alone suffice to align agents in time-critical cooperative tasks.
---
*Auto-collected on 2026-05-27*