Summary
LatentToM (Latent Theory of Mind), presented at CoRL 2025 as an Oral paper, tackles decentralized multi-robot collaboration where each robot has its own camera and compute unit. Each robot maintains two latent-space representations: a self-embedding (its own observations and state) and a consensus embedding (a shared understanding of the scene). The key innovation is a sheaf-theory-based loss that keeps consensus embeddings consistent across different viewpoints, recovering global consistency from consistent local observations. Execution can be fully decentralized—robots infer each other's intentions by observing actions without any communication—or robots can share consensus embeddings to speed up cooperation. In dual-arm manipulation experiments, LatentToM matches centralized policies in performance while being naturally robust when robots suffer temporary failures or communication delays, scenarios in which centralized policies collapse entirely. The core insight: good teamwork does not require every member to know everything—only a consistent shared picture.
Multi-robot collaboration is a long-standing challenge: when each robot has its own camera and compute unit, how do they know what the others are "thinking"?
LatentToM (Latent Theory of Mind), an Oral paper at CoRL 2025, proposes an approach in which each robot maintains two latent-space representations:
- Self-embedding — what the robot sees and its own state
- Consensus embedding — the shared understanding of the scene's state
The key innovation is a Sheaf Theory-based loss that ensures the consensus embeddings stay consistent across different viewpoints—recovering global consistency from consistent local observations.
Key points
- Two latent spaces per robot: self-embedding (ego view + own state) and consensus embedding (shared scene understanding)
- Sheaf theory loss enforces viewpoint-consistent consensus embeddings
- Execution can be fully decentralized: no communication—robots infer each other's intentions by observing their actions
- Alternatively, robots can share consensus embeddings to accelerate collaboration
- In dual-arm manipulation, performance is on par with centralized policies
- Naturally robust to temporary robot failures or latency, where centralized policies break down entirely
Core insight: Good teamwork doesn't require every member to know everything—just a consistent "shared picture."
Reference: [LatentToM / CoRL 2025 Oral]
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177619966