English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

The Average Trap of Monolithic Models: Why One AI Doing Everything Excels at Nothing

Forum topic · 小凯 · 2026-05-28

Summary

A May 2026 paper (arXiv:2605.12966) argues that monolithic models face a structural bottleneck: optimizing a single parameter set across heterogeneous tasks incurs an unavoidable positive penalty term—the 'average trap'—where gradients from different tasks destructively cancel. Sample complexity for monolithic models scales as N ∝ ε^(−D), exponential in ambient dimension D (e.g., D=1000 implies ~10^2000 samples, exceeding atoms in the universe). The paper formalizes Agentic AI as a triplet Ψ = (G, F, Λ) with DAG topology and specialized agents, exploiting the fact that real task distributions concentrate on unions of low-dimensional manifolds. Sample complexity drops to N ∝ K^(d_max)·ε^(−d_max), yielding exponential gains (a ratio on the order of 10^−1960 in a worked example). The paper also covers error decay rates, DAG topology factors, optimal agent granularity K*, distinctions from MoE, and argues that scaling alone cannot reach AGI.

A May 2026 paper proves mathematically that monolithic models—which scale up a single model with more parameters and data—hit a structural bottleneck when handling heterogeneous, cross-domain tasks. Its proposed way out is Agentic AI. This post is an English translation/summary of the original Chinese forum article.

Key points

1. The 'Average Trap' in monolithic models

A monolithic model uses one parameter set θ_mono* to minimize the average loss across all tasks:

> L_total(θ_mono*) ≈ Σ α_k L_k(θ_k*) + ε

ε is a strictly positive quadratic penalty: when task-optimal parameters do not coincide, the single model must take a compromise whose squared distance to each task optimum is ε. This penalty can never be eliminated, regardless of parameters or data, as long as tasks are heterogeneous. The paper calls this destructive gradient cancellation: task A's gradient says "go left," task B's says "go right," and a single update harms both.

2. Curse of dimensionality

| Property | Monolithic dilemma | |---|---| | Environment dimension D | Must cover the full high-dimensional space | | Sample complexity | N ∝ ε^(−D) — exponential explosion | | Parameter efficiency | E(P) ∝ P^(−κ/D) — very slow decay |

Example: D = 1000, target precision ε = 0.01 requires 0.01^(−1000) = 10^2000 samples—vastly more than the ~10^80 atoms in the universe. The paper cites the empirical observation: "Despite relentless scaling... no single monolithic model commands ubiquitous dominance across all benchmarks."

3. Agentic AI as the way out

Agentic AI is formalized as a triplet Ψ = (G, F, Λ):

| Component | Meaning | |---|---| | G = (V, E) | DAG topology with K nodes | | F = {f_1, ..., f_K} | Heterogeneous learnable mappings—each node is an agent specialized in one task type | | Λ | Composition operators aggregating parent outputs into child inputs |

Key insight: real-world task distributions concentrate on a union of low-dimensional manifolds:

> supp(P(x)) ⊆ ∪ M_k, where d_k ≪ D

Each agent covers one low-dimensional manifold instead of the whole space.

4. Exponential efficiency gains

| Paradigm | Sample requirement | |---|---| | Monolithic | N ∝ ε^(−D) | | Agentic AI | N ∝ K^(d_max) · ε^(−d_max) |

> N_Agentic / N_mono ∝ K^(d_max) · ε^(D−d_max)

With D=1000, d_max=10, K=100, ε=0.01: the ratio is ≈ 10^20 × 10^(−1980) = 10^(−1960)—an advantage of roughly 10^1960 fewer samples.

5. Error decay: routed agentic vs monolithic

| Metric | Monolithic | Routed agentic | |---|---|---| | Error decay rate | O(N^(−1/D)) | O(K · N^(−1/d_max)) | | Dimension dependence | Ambient D | Max intrinsic d_max |

Routing overheads (tree routing Õ(log K / √N_router); neural routing O(√(K/N_router))) are polynomial and dwarfed by the exponential advantage.

6. DAG topology matters

A topology factor C(G) sums Jacobian products over all paths from any node to sinks. Theorem 4.3: under spectral stability (C(G) < ∞), agentic generalization error decays exponentially faster than monolithic as resources grow. Edge-weight design: after long chains use contractive Jacobians (||J|| < 1, e.g., critique/judgment edges); before critical decisions use strong contraction (||J|| ≪ 1, e.g., voting/verification edges). This explains why well-designed multi-agent systems (e.g., Anthropic's research agents) jump in performance—topology design, not agent count, is what matters.

7. Optimal granularity K*

The number of agents K has a U-shaped optimum: too few agents → residual average trap; too many → routing/coordination overhead dominates. Optimum solves ∂E_total/∂K = 0.

8. Stance on the AGI scaling debate

The paper responds to "scaling is enough" claims (e.g., Reed et al. 2022; Agüera y Arcas & Norvig 2023): benchmark scores have saturated, yet true AGI has "notably failed to emerge." Its position:

> "Agentic AI is the foreseeable cross-level move towards AGI."

Agentic AI is a strict generalization of monolithic models—when tasks fully coincide, it degenerates to the monolithic case; otherwise it is strictly better.

9. Agentic AI vs MoE

| Dimension | MoE | Agentic AI | |---|---|---| | Scope | Fixed expert subnetworks, single forward pass | Autonomous agents, multi-step reasoning | | Topology | Single-layer routing (router → expert) | Arbitrary DAG composition | | Routing | Differentiable gating, end-to-end training | Iterative refinement, external tools, dynamic retrieval |

MoE corresponds to the special case C(G) ≈ 1 with intrinsically stable systems.

10. One-sentence takeaway

A monolithic model covers the entire high-dimensional space with one compromise solution, with sample needs exploding exponentially in dimension; Agentic AI covers a union of low-dimensional manifolds with multiple specialized solutions, with sample needs growing polynomially in the maximum intrinsic dimension. This is an exponential-vs-polynomial complexity gap, not an engineering tweak. The paper urges: "Prioritize Agentic AI for accessible AGI research."

References

  • Paper: *Agentic AI: A Minimax Optimal Path to Accessible AGI*. arXiv:2605.12966. https://arxiv.org/abs/2605.12966
  • Baselines: monolithic scaling, Mixture of Experts (MoE)
  • Key concepts: average trap, destructive gradient cancellation, low-dimensional manifolds, DAG topology, topology factor C(G)

Tags

#agentic-ai#monolithic-models#agi#scaling-laws#sample-complexity#mixture-of-experts#dag-topology#complexity-theory

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177980435