Locally Coherent, Globally Incoherent: Multi-Agent LLM Systems and the Probability Theory Trap

Forum topic · 小凯 · 2026-05-30

Summary

A Chinese tech forum post analyzes a May 2026 arXiv paper (arXiv:2605.30335) by independent researcher Anany Kotawala, 'Locally Coherent, Globally Incoherent: Bounding Compositional Incoherence in Multi-Component LLM Agents.' The paper quantifies a fundamental flaw in multi-agent AI architectures: when N models each assess a partial view of a problem and their probabilistic judgments are merged, the combined assertions can violate basic probability axioms. Across 1,876 real multi-model ensemble tests on four mid-size LLMs, 33%-94% of combinations were probabilistically incoherent, with an average expected loss of +0.115 nats per bet under a gambling interpretation. The paper introduces the compositional residual epsilon-star (L2 distance from the coherent probability polytope), a product-structure dichotomy characterizing when local coherence guarantees global coherence, and a Rayleigh-quotient formula predicting residuals within 7% error. Three intuitive fixes—retrieval augmentation, partition-aware prompting, and aggregator LLMs—all failed or regressed. A mathematically sound fix via Boyle-Dykstra projection exists but requires exact coupling constraints. Implications for high-stakes multi-agent deployments are discussed.

Key points

A Chinese tech forum post (zhichai.net) presents an in-depth analysis of a paper on probabilistic incoherence in multi-agent LLM systems, framed by a doctor analogy: three doctors each examine a different region of an X-ray and report independent pneumonia probabilities (60%, 50%, 40%). Averaging them yields 50%, but the evidence fragments are correlated—so naive merging can produce assertions that violate probability axioms entirely.

The paper

| Item | Content | |------|---------| | Title | Locally Coherent, Globally Incoherent: Bounding Compositional Incoherence in Multi-Component LLM Agents | | Author | Anany Kotawala (independent researcher) | | arXiv ID | arXiv:2605.30335 | | Submitted | May 28, 2026 | | Category | cs.AI |

Core findings

Compositional incoherence is routine, not theoretical. Each agent estimates a conditional probability, e.g. $P(X \mid E_1)$ and $P(X \mid E_2)$ . Merging these assumes the evidence sets are independent for $$X$$ , but they are coupled in the joint distribution. Ignoring the coupling can yield probabilities above 100%, mutually exclusive events whose probabilities don't sum to 1, or $P(A \cap B) > P(A)$ .
Quantification via the compositional residual $\varepsilon^*$ : the $$L^2$$ distance between the merged assertions and the coherent probability polytope (the convex set of joint distributions satisfying the probability axioms). It is computable at runtime from system outputs plus declared cross-component coupling constraints.
Scale of the problem: across 1,876 ensemble cliques built from four mid-size LLMs, 33%–94% of combinations had $\varepsilon^* > 0$ , depending on relation class and component count. Interpreted as betting, incoherent combinations yield an average expected loss of +0.115 nats per bet—meaning an adversary could systematically profit from these contradictory probability judgments.
Product-structure dichotomy: local coherence guarantees global coherence if and only if the problem partition satisfies a product structure (no informational overlap between components' evidence). Most real-world problems cannot be cleanly partitioned, so most deployed multi-agent systems inherit the flaw.
Rayleigh-quotient prediction: in three of four relation classes, predicted residuals match measured residuals within 7% error, enabling pre-deployment risk prediction.

Why intuitive fixes failed

1. Retrieval augmentation — giving each component more context: failed; residuals did not meaningfully drop, because the coupling structure remained. 2. Partition-aware prompting — telling each component about its partition's limits: failed; awareness of limited view did not translate into probabilistically correct calibration. 3. Aggregator LLM — a dedicated LLM merges outputs: regressed; residuals increased, as the aggregator exhibited self-anchoring drift, endorsing rather than resolving the contradictions.

Mathematical fix vs. engineering reality

The paper proposes hierarchical Boyle-Dykstra projection: deterministically project incoherent merged assertions back onto the coherent probability polytope (nearest coherent distribution in $$L^2$$ ). It converges and is computable.
The catch: the projection requires explicitly declared coupling constraints. If you knew the exact coupling structure, you likely wouldn't have produced the incoherence in the first place—this is precisely the global knowledge multi-agent systems are designed to avoid needing.
A complementary tool is an anytime-valid e-process for continuously monitoring $\varepsilon^*$ at runtime and alerting when the residual exceeds a threshold.

Open questions (raised by the forum author)

Single-author study with four mid-size models; frontier-model re-validation details (Section 5.5) are not disclosed in the abstract—generalization is unconfirmed.
No engineering-actionable threshold for $\varepsilon^*$ : how large is too large for real decisions?
Whether coupling constraints could be estimated data-driven for projection-based repair—and whether that estimation is as hard as the original problem.
Whether larger or better-trained models could overcome the three failed fixes, or whether information asymmetry across partitions is an information-theoretic hard limit.

Takeaway

The paper's deeper message: splitting a judgment and stitching it back does not necessarily restore the original shape. Split + parallel + merge may produce mathematically self-contradictory results—not because models are weak, but because the distribution of information itself creates probabilistic incoherence. For high-stakes multi-agent deployments (medical, financial, safety-critical), not knowing your system's $\varepsilon^*$ means not knowing what you are betting on.

References

1. Kotawala, "Locally Coherent, Globally Incoherent: Bounding Compositional Incoherence in Multi-Component LLM Agents", arXiv:2605.30335, 2026. 2. Boyle & Dykstra, "A Method for Finding Projections onto the Intersection of Convex Sets in Hilbert Spaces", 1986. 3. Ramdas et al., "Admissible Anytime-Valid Sequential Inference", Statistical Science, 2023. 4. de Finetti, "Theory of Probability", Wiley, 1974. 5. Park et al., "Generative Agents: Interactive Simulacra of Human Behavior", UIST 2023.