Locally Coherent, Globally Incoherent: The Probability Trap of Multi-Agent LLM Systems
> Three doctors each examine a different part of an X-ray. > > Doctor A says: "There's a shadow on the left side — the probability of pneumonia is 60%." Doctor B says: "There's fluid on the right side — 50%." Doctor C says: "The trachea is shifted — 40%." > > What is the *combined* assessment? If you average the three probabilities, you get 50%. Reasonable, right? > > But look closer: each doctor saw only part of the evidence. A's left-side shadow and B's right-side effusion may be correlated — pneumonia often affects both lungs. C's tracheal shift may overlap with A's shadow. These clues are not independent. Naively merging them as independent probabilities can violate the basic axioms of probability theory — for instance, the computed probabilities of "pneumonia" and "not pneumonia" no longer sum to 100%. > > In May 2026, Anany Kotawala published a single-author paper on arXiv that quantifies this problem with a rigorous mathematical framework. The conclusion is unsettling: across 1,876 real multi-model composition tests, 33% to 94% of compositions violated basic axioms of probability theory. Worse — three intuitively sensible fixes all failed or backfired.
| Item | Detail | |------|--------| | Paper | Locally Coherent, Globally Incoherent: Bounding Compositional Incoherence in Multi-Component LLM Agents | | Author | Anany Kotawala (independent researcher) | | arXiv ID | 2605.30335 | | Submitted | May 28, 2026 | | Category | cs.AI | | Core finding | Multi-component LLM agent systems violate probability axioms in 33%–94% of real scenarios; three intuitive fixes (retrieval augmentation, partition-aware prompting, aggregator LLM) all fail or regress |
1. Why stitched-together judgments fall apart
Multi-agent systems work like this: a problem is split into N parts, N models each handle their piece, and N judgments are merged into a final answer.
The problem lies in the "merge" step.
One agent estimates "given the evidence I saw, what's the probability of X" — a conditional probability \(P(X \mid E_1)\). Another estimates \(P(X \mid E_2)\). Merging these directly assumes \(E_1\) and \(E_2\) influence X independently — but in the joint probability distribution they are coupled. Ignoring that coupling, the merged "probability" can violate the fundamental axioms: an event with probability over 100%, mutually exclusive events not summing to 100%, or P(A and B) exceeding P(A).
These are mathematically impossible — yet in multi-agent merging, they actually happen. Kotawala's 1,876 real tests show this is not a theoretical possibility but a routine practice.
2. A measuring stick called ε*
The paper's core contribution is not discovering the problem — quantifying it.
Kotawala defines the compositional residual \(\varepsilon^*\): the \(L^2\) distance between the composed probability assertions and the "self-consistent probability polytope" — the convex polytope of all joint distributions satisfying the probability axioms. Your system's composed assertions either fall inside the polytope (coherent) or outside (\(\varepsilon^* > 0\)). It can be computed at runtime from the system's outputs and the declared cross-component coupling constraints.
The experimental results are stark. Across 1,876 "ensemble cliques" of four mid-sized LLMs, the fraction with \(\varepsilon^* > 0\) ranged from 33% to 94%, depending on relation class and component count. In the worst case, nearly every composition was probabilistically absurd.
Quantified as gambling: treating the agents' probability assertions as betting odds, incoherent compositions produced an average expected loss of +0.115 nats per bet. If you use these incoherent agent judgments to guide decisions, an adversary can systematically profit from you, because your beliefs are logically self-contradictory.
3. When is local coherence enough?
The paper identifies a clean mathematical condition: the product-structure dichotomy. If the joint problem's coupling structure satisfies a "product property" — intuitively, no information overlap between components' evidence — then local coherence automatically guarantees global coherence. If not, residual appears.
Concretely: whether your multi-agent system is safe depends on how you split the problem. Clean, non-overlapping information boundaries — safe. Crossing boundaries — unsafe. Most real problems are not organized along product structures, so most deployed multi-agent systems carry this probabilistic flaw by construction.
The paper also provides a Rayleigh-quotient prediction formula: for three of four relation classes, it predicts the measured compositional residual within 7% error — letting you anticipate failure modes before deployment.
4. Why the intuitive fixes all failed
Fix 1: Retrieval augmentation. Give each component more context to reduce information asymmetry. Failed — residual did not significantly drop; extra information didn't change the coupling structure.
Fix 2: Partition-aware prompting. Tell each component which partition it handles and what evidence lies elsewhere. Failed — knowing about a limited view does not automatically produce probabilistically correct restraint.
Fix 3: Aggregator LLM. Replace mechanical merging with an LLM that reads all component outputs. Regressed — the aggregator increased residual, suffering "self-anchoring drift": led by the components' hallucinated assertions, it stamped contradictions with an aura of authoritative consistency.
5. Mathematical fix vs. engineering fix
The mathematically correct solution is a layered Boyle-Dykstra projection: project the incoherent composition onto the nearest self-consistent distribution in the polytope. It is deterministic, computable, and convergent.
But it has an engineering problem: the projection imposes a coupling-structure assumption. It requires you to declare the coupling constraints among components. If the constraints are accurate, the projection is accurate; if not, the result is coherent but factually wrong.
This yields an awkward bind: if I knew the exact coupling relationships, I wouldn't have committed the composition error in the first place. Precise coupling knowledge is exactly what multi-agent systems were designed to avoid needing.
The paper also contributes an anytime-valid e-process for continuously monitoring compositional coherence at runtime, raising alarms when residual exceeds a threshold.
6. Honest uncertainties
- Single-author validation scale: experiments used four mid-scale models with 1,876 tests, plus a frontier-model re-check (Section 5.5) whose details are not disclosed in the abstract. If frontier models diverge, the conclusion's scope narrows.
- **Operational threshold for ε*: 33%–94% is a wide range. How large must the residual be before it matters for real decisions? The gambling experiment (+0.115 nats/bet) offers a lens, but mapping nats to actual loss depends on bet size.
- Deployability of Boyle-Dykstra: it needs precise coupling constraints. Could constraints be estimated from data — or is estimating them as hard as the original problem?
- Are the three failures surmountable? Would bigger models or specialized architectures help, or is asymmetric coupling information an information-theoretic hard limit that compute cannot break?
7. The whole is not the sum of its parts
The deep message: split a judgment apart and stitch it back, and you may not get the original shape back.
This challenges a default assumption. Multi-agent systems, ensemble reasoning, federated judgment — all rest on the belief that split + parallel + merge equals and improves. The paper says: split + parallel + merge = possibly a violation of probability theory.
This isn't an argument against multi-agent systems. It's a warning that you've introduced a new error source — not model capability, not training data, but the distribution of information itself: who sees what, and who doesn't, creating a probabilistic incoherence that was never systematically tracked before.
Kotawala turned "something feels off" into something measurable, testable, and fixable — if you honestly declare which information is coupled across components.
If your multi-agent system is making a high-stakes judgment — medical, financial, safety-critical — and you don't know the value of \(\varepsilon^*\), you don't know what you're betting on.
---
| Item | Detail | |------|--------| | Paper | Locally Coherent, Globally Incoherent: Bounding Compositional Incoherence in Multi-Component LLM Agents | | Author | Anany Kotawala (independent) | | arXiv ID | 2605.30335 | | Key contributions | (1) Quantifies multi-agent probabilistic coherence via the compositional residual — 33%–94% incoherence across 1,876 tests; (2) product-structure dichotomy for when local coherence implies global coherence; (3) Rayleigh-quotient residual prediction within 7% error; (4) deterministic Boyle-Dykstra projection repair; (5) anytime-valid e-process monitoring; (6) three intuitive LLM-side fixes all fail, revealing the problem's depth | | Limitations** | Single-author experiments; frontier-model revalidation details undisclosed; no engineering-actionable threshold for ε*; projection requires precise coupling constraints whose acquisition may be as hard as the original problem; whether distributed information coupling is an information-theoretic hard limit remains open |
References: 1. Kotawala, "Locally Coherent, Globally Incoherent: Bounding Compositional Incoherence in Multi-Component LLM Agents", arXiv:2605.30335, 2026. 2. Boyle & Dykstra, "A Method for Finding Projections onto the Intersection of Convex Sets in Hilbert Spaces", 1986. 3. Ramdas et al., "Admissible Anytime-Valid Sequential Inference", Statistical Science, 2023. 4. de Finetti, "Theory of Probability", Wiley, 1974. 5. Park et al., "Generative Agents: Interactive Simulacra of Human Behavior", UIST 2023.