DR.WELL: Teaching Embodied LLM Agents to Collaborate via Symbolic World Models

Forum topic · ✨步子哥 · 2025-11-08

Summary

DR.WELL (Dynamic Reasoning and Learning with a Symbolic World Model for Embodied LLM-Based Multi-Agent Collaboration), proposed by researchers from USC and Carnegie Mellon (arXiv:2511.04646), is a neurosymbolic framework that coordinates LLM-driven embodied agents without step-by-step trajectory synchronization. Instead of forcing agents to align every action, DR.WELL lets them negotiate roles through a structured two-round protocol (proposal and commitment with quorum constraints), then plan privately using a compact vocabulary of symbolic actions such as MOVETOBLOCK, RENDEZVOUS, PUSH, and YIELDFACE. A dynamic symbolic world model—a four-layer graph of episodes, tasks, plan prototypes, and instances—records success statistics, externalizes shared experience, and guides both negotiation and plan refinement. In the CUBE cooperative box-pushing benchmark, where boxes require multiple agents pushing simultaneously, DR.WELL significantly outperformed zero-shot baselines: higher completion rates, fewer environment steps over time, and stable task division, at a modest wall-clock cost from negotiation. The authors attribute the gains to symbolic-level abstraction, externalized collective memory, minimal high-signal communication, and closed-loop self-refinement. Future directions include partial observability, runtime renegotiation, intra-team tactical communication, and probabilistic symbolic models. The framework illustrates how explainable, decentralized, and learnable multi-agent collaboration can emerge from structured consensus and shared symbolic memory rather than heavier neural networks.

This post is a detailed Chinese-language analysis of DR.WELL (arXiv:2511.04646v1), a framework by Nourzad, Yang, Chen, and Joe-Wong (USC / CMU) for embodied multi-agent collaboration driven by large language models. Below is a structured English summary of the full discussion.

Key points

Motivation: LLM-based agents are brittle to prompt wording [Shah et al., 2025], and classic MARL is costly and poorly generalizable. Traditional coordination either deadlocks or drowns in unstructured chat. DR.WELL builds on neurosymbolic AI: LLMs provide flexible perception/reasoning, while an explicit symbolic world model provides shared, human-readable structure.
Two-round negotiation protocol:

Proposal phase

Block_2: 54.5% success, optimal team size = 1

Commitment phase

consensus and quorum

Symbolic planning with self-refinement: After committing, each agent independently drafts a plan as a sequence of parameterized symbolic actions (e.g., MoveToBlock -> Rendezvous -> Push), then refines it against the world model's ranked plan prototypes and instances with success rates. No central controller is needed.
Dynamic world model: A symbolic graph G = (V, E) with four node layers — episodes, tasks, plan prototypes, plan instances — updated incrementally (G_{k+1} = G_k ∪ ΔG_k). Outcomes propagate upward, aggregating statistics and enabling continual learning without catastrophic forgetting. It acts as collective memory: failures of one agent become lessons for all.
CUBE benchmark: A grid world where a box of size w × w has weight w and requires w agents pushing the same face simultaneously. The action vocabulary has five primitives: WAITAGENTS, RENDEZVOUS, MOVETOBLOCK, PUSH, YIELDFACE. Both tensor and symbolic observations are provided.

Findings

Zero-shot baselines ("push the nearest box") show binary completion patterns, flat timing curves, and crowding on single-agent tasks — a local-optimum trap of individually rational, collectively irrational agents.
DR.WELL achieves near-complete task completion after early episodes, steadily decreasing environment steps, and stable division of labor by episode 5; wall-clock time rises slightly due to negotiation overhead, trading meeting time for execution efficiency.
World-model visualizations show the graph evolving from sparse, failure-dominated (episode 1) to a dense, success-dominated network (episode 10) — an interpretable strategic handbook rather than a black box.

Why it works

1. Symbolic abstraction: aligning intentions ("push Block_3") instead of raw trajectories avoids cascading micro-conflicts. 2. Externalized experience: shared memory beats experience locked in individual weights. 3. Minimal communication: two structured rounds at key moments, not free-form chat. 4. Closed-loop refinement: draft → world-model refinement → execution → results written back.

Future directions

Sub-goal reasoning for implicit steps, partial observability, runtime renegotiation after failures, intra-subtask tactical communication, and probabilistic symbolic models for uncertainty.

References cited in the post

1. Nourzad, N., Yang, H., Chen, S., & Joe-Wong, C. (2025). *DR. WELL: Dynamic Reasoning and Learning with Symbolic World Model for Embodied LLM-Based Multi-Agent Collaboration*. arXiv:2511.04646v1. 2. Garcez, A. S. D., & Lamb, L. C. (2023). *Neurosymbolic AI: The 3rd Wave*. arXiv:2012.05876. 3. Shah, D., Osinski, B., Levine, S., et al. (2025). *LMPC: Learning Model Predictive Controllers for LLM-Based Agents*. ICLR. 4. Yang, H., Chen, S., & Joe-Wong, C. (2023). *LLM-Based Multi-Agent Reinforcement Learning: A Survey*. NeurIPS Workshop. 5. Mao, J., Gan, C., Kohli, P., et al. (2025). *Concept Learners for Generalizable Neurosymbolic Agents*. CoRL.