Learning Coordinated Preference for Multi-Objective Multi-Agent Reinforcement Learning (PCMA)

Forum topic · 小凯 · 2026-06-16

Summary

A June 2026 arXiv paper (2606.14693) by Pengxin Wang, Lihao Guo, and Yi Xie introduces PCMA, Preference Coordinated Multi-agent Policy Optimization, a method for cooperative multi-objective multi-agent reinforcement learning (MOMARL). MOMARL models team decision making under multiple potentially conflicting objectives, where conflicts arise both across objectives and across agents with different observations, roles, and contributions. PCMA learns coordinated agent-specific preferences so that agents make complementary trade-offs rather than optimizing uniformly. The authors formulate cooperative MOMARL as a team-optimal game and prove that, under suitable conditions, preference diversity can induce team improvement via a first-order improvement decomposition. Experiments across multiple cooperative MOMA environments and a practical traffic-signal control scenario show that PCMA improves both performance and the coordination of trade-offs among objectives and agents.

Paper Overview

Field: Multi-Agent (MA) reinforcement learning
Authors: Pengxin Wang, Lihao Guo, Yi Xie
Published: 2026-06-12
arXiv: 2606.14693

Abstract

Cooperative multi-objective multi-agent reinforcement learning (MOMARL) models team decision making under multiple, potentially conflicting objectives. In this setting, conflicts arise not only across objectives but also across agents with different observations, roles, and contributions. We propose Preference Coordinated Multi-agent Policy Optimization (PCMA), which learns coordinated agent-specific preferences to enable complementary trade-offs among agents. Theoretically, we formulate cooperative MOMARL as a team-optimal game and show that, under suitable conditions, preference diversity can induce team improvement through a first-order improvement decomposition. Experiments on multiple cooperative MOMA environments and a practical traffic-control scenario show that PCMA improves both performance and trade-off coordination.

Key Contributions

PCMA method: learns coordinated, agent-specific preferences so individual agents pursue complementary trade-offs across objectives instead of identical preferences.
Theoretical foundation: cooperative MOMARL is formulated as a team-optimal game; a first-order improvement decomposition shows when preference diversity leads to team-level improvement.
Empirical validation: improvements in performance and trade-off coordination across multiple cooperative MOMA benchmark environments and a real-world traffic-control scenario.