Paper Overview
- Field: Machine Learning (Reinforcement Learning Benchmarks)
- Authors: Derin Gezgin, Jim O'Connor, Tanner Goodwin
- Posted: 2026-08-11
- arXiv: 2508.03798
- Real-time combat
- High-dimensional visual input
- Sparse terminal rewards
- Random policy
- Expert system
- Evolutionary baseline
- PPO (Proximal Policy Optimization) trained from visual input
- DQN (Deep Q-Network) trained from visual input
- The expert system and evolutionary baseline each defeat the Asylum Demon tutorial boss, with peak win rates of 63% and 43% respectively.
- None of the five methods defeat the remaining four DSLE-5 bosses.
- PPO and DQN show no measurable learning within wall-clock budgets of dozens of hours per run:
- At most 0.33% win rate on the tutorial boss
- 0% win rate on the other four bosses
- A broader study running the evolutionary baseline across all 22 encounters under favorable full-50-stat builds wins only a few additional early-game bosses, with all remaining encounters lost.
- Deaths in under 10 seconds in cramped, multi-target encounters
- Stalemates lasting nearly one minute with negligible damage inflicted
Summary
The authors introduce the Dark Souls Learning Environment (DSLE), a containerized benchmark platform that exposes all 22 boss encounters of *Dark Souls: Remastered* to game-playing agents through a Gymnasium-style interface. DSLE combines three challenging properties for RL research:
Each environment step corresponds to a real action executed against the running game, rather than a simulator abstraction.
DSLE-5 Benchmark Subset
To support controlled comparison, the authors define DSLE-5, a representative five-boss subset covering distinct combat archetypes:
1. Melee fight 2. Spatially constrained arena 3. Environmental-hazard fight 4. Multi-target fight 5. Fast final-boss fight
DSLE-5 is recommended as the starting suite for agents built on DSLE.
Evaluated Methods
On DSLE-5, five approaches were evaluated:
Key Results
Failure Modes
The authors report failures not only via win rate but also via survival time and damage dealt. Failure cases include:
Conclusion
DSLE provides a challenging, real-game benchmark for reinforcement learning research, and current standard methods (PPO, DQN, evolutionary baselines) are far from solving it, even on a reduced five-boss suite.
--- *Auto-collected on 2026-08-12*