LMAC: Using LLMs as Communication Protocol Designers for Multi-Agent RL

Forum topic · 小凯 · 2026-05-19

Summary

In cooperative multi-agent reinforcement learning (MARL), communication is the key to resolving partial observability, yet existing approaches either exchange information inefficiently (all agents broadcasting everything) or transmit insufficient information (fixed, predefined message formats encoding only a subset of local observations). LMAC, by Bae, Park, Lee, and Han (ICML 2026), uses an LLM as a communication protocol designer. The core idea is to leverage the LLM's reasoning ability to iteratively design a protocol aimed at helping all agents reconstruct the underlying true state as accurately and consistently as possible. The LLM applies a state-awareness criterion to judge whether the current protocol is sufficient, and if not, improves it by specifying which information agents should transmit to close their knowledge gaps. On multi-agent benchmarks, LMAC improves cross-agent state reconstruction and delivers substantial performance gains over existing communication baselines. The post also raises open questions about LLM inference latency for real-time decision-making, whether protocol design happens offline during training or online at inference, and how the LLM is integrated.

LLM-Guided Communication for Cooperative Multi-Agent RL

In multi-agent reinforcement learning (MARL), communication is the key to solving partial observability — each agent only sees its own local observation, and the global state must be recovered through communication. Existing methods either suffer from inefficient information exchange (all agents broadcast everything) or transmit too little information (predefined, fixed message formats that encode only a subset of local observations).

What LMAC Does

Bae, Park, Lee, and Han (ICML 2026) propose LMAC, which uses an LLM as a communication protocol designer. The core idea: leverage the LLM's reasoning ability to iteratively design a communication protocol, with the goal of enabling all agents to reconstruct the underlying true state as accurately and consistently as possible.

The LLM uses a state-awareness criterion to judge whether the existing protocol is sufficient.
If it is not sufficient, the LLM improves it — telling agents which information they should transmit to eliminate knowledge gaps between them.

Results

On multi-agent benchmarks, LMAC improves state reconstruction across agents and delivers substantial performance gains over existing communication baselines.

Open Questions (from the original author)

Does LLM inference latency affect the real-time nature of communication, especially in MARL scenarios requiring millisecond-level decisions?
Is the iterative protocol design process completed offline during training, or run online at inference time?
Is the LLM embedded as a component in LMAC, or does it drive protocol design via API calls?

References

1. Bae, S., Park, Y., Lee, S., & Han, S. (2026). *LLM-Guided Communication for Cooperative Multi-Agent Reinforcement Learning*. arXiv:2605.18077 [cs.AI]. 2. Foerster, J., et al. (2016). *Learning to Communicate with Deep Multi-Agent Reinforcement Learning*. NeurIPS. 3. Singh, A., et al. (2019). *Learning to Communicate with Neural Intermediates*. AAAI.