DNQ: Deep Nash Q-Networks for Partially Observable n-Player Games

Forum topic · 小凯 · 2026-06-06

Summary

This paper (arXiv:2506.08262) by Qintong Xie, Edward Koh, and Xavier Cadet introduces DNQ, a solver-in-the-loop equilibrium supervision framework for training agents in partially observable multi-player games. The authors use multi-turn simultaneous bidding as a controlled testbed for broader competitive systems—such as auctions, resource allocation, and security competitions—where multiple decision-makers act simultaneously under shared constraints, limited information, and repeated interactions. DNQ alternates between four stages: trajectory collection, critic-based payoff estimation, equilibrium computation, and policy imitation. A key contribution is a pairwise formulation that substantially reduces equilibrium-solving cost and training time, and experiments demonstrate that the pairwise approach scales to larger numbers of agents. The work falls within machine learning and was released on June 11, 2025.

Paper Overview

Research Area: ML Authors: Qintong Xie, Edward Koh, Xavier Cadet Released: 2025-06-11 arXiv: 2506.08262

Key Idea

Many real-world competitive systems require multiple decision-makers to act simultaneously under shared constraints, limited information, and repeated interactions—examples include auctions, resource allocation, and security competitions.

The authors adopt multi-turn simultaneous bidding as a controlled testbed for this class of problems and propose DNQ, a solver-in-the-loop equilibrium supervision framework for training bidding agents.

How DNQ Works

DNQ alternates between:

Trajectory collection — gathering interaction data from the bidding environment
Critic-based payoff estimation — evaluating returns with learned critics
Equilibrium computation — solving for equilibria as supervision targets
Policy imitation — training the agents' policies toward the computed equilibria

Contributions

A pairwise formulation that significantly reduces equilibrium-solving cost and training time
Experimental evidence that the pairwise approach scales to more agents

---

Auto-collected on 2025-06-11