> Introduction: In 2026, amid the flood of trillion-parameter sparse MoE models and benchmark-chasing, a core puzzle persists: why does Claude Fable—frequently surpassed by specialized models on individual capabilities—still command a startling dominance in global reasoning, unfamiliar-architecture analysis, and long-horizon exploration? This article deconstructs the data-hunger asymmetry between Dense and MoE architectures from first principles of compute economics, representation topology, and cognitive psychology, and proposes a physical account of large-model general intelligence (the G-factor).
Key points
- Dense compute rigidity: A dense Transformer's training cost follows C ≈ 6ND (N = total parameters, D = token count). Every parameter participates in every token's forward/backward pass, so an under-trained oversized dense model is economically unviable—market-surviving dense flagships must be fully saturated with high-quality, high-density pretraining data.
- MoE's triple data-hunger trap: 1. *Exposure dilution*: per-expert token exposure is D_expert = kD/E, so per-expert exposure density ρ_expert = k · (D/N_total). Adding experts barely increases GEMM compute, incentivizing inflation of total parameters far beyond what the high-quality data pool can support. 2. *Gradient noise*: Top-k routing is non-differentiable (softmax + noise or RL approximations), and load-balancing auxiliary losses act as optimization perturbations unrelated to the causal modeling objective, degrading gradient SNR. 3. *Representation fragmentation*: Dense parameters form a globally smooth, connected Riemannian manifold; MoE experts are disjoint local subspaces stitched by gating—the physical boundaries between experts create irreducible "cognitive impedance" for high-order cross-domain reasoning.
- G-factor vs. S-factor (via Spearman's two-factor theory): S-factor abilities are narrow, driven by deterministic oracles (compilers, unit tests, answer matching), and easily saturated by task-specific RL—ideal for benchmark maxing. G-factor abilities are a self-organized cognitive skeleton formed without explicit reward, in featureless open-ended domains, resistant to gaming.
- The Fable phenomenon: Claude Fable 5's surprising strength in frontier mathematics (e.g., Jacobi conjecture counterexample construction, high-dimensional PDE analysis) does not stem from injected theorem corpora. Long-horizon agentic training (perceive → hypothesize → call tools → hit errors → self-refute → adjust search) distills three meta-capabilities: long-range symbolic reasoning, search-space intuition, and rigorous self-refutation. Mathematical prowess is the natural projection of a dense general manifold onto formal systems.
- The "Claude Dense hypothesis" rests on three behavioral fingerprints: 1. Premium pricing with relatively low throughput, consistent with full-parameter GEMM costs; 2. Complete absence of parameter-count marketing (dense parameter numbers would look small); 3. A knowledge–reasoning scissors gap: occasionally fuzzy long-tail factual recall, but never a collapse of logical structure on unfamiliar complex tasks.
- System philosophy: Factoid knowledge can be fetched at low latency via tool use, code interpreters, search, and RAG. Therefore the base model's entire parameter budget should be staked on irreplaceable endogenous reasoning depth (G-factor).
Industrial decision matrix
| Architecture | Strengths | Fatal weaknesses | Best-fit positioning | | :--- | :--- | :--- | :--- | | Massive sparse MoE | Very low inference cost, high throughput, strong long-tail factual retrieval | Representation fragmentation, gradient noise, brittle cross-domain logic on unfamiliar tasks | Consumer Q&A, customer service, multilingual translation, high-concurrency APIs | | Fully-saturated Dense | Globally coherent representations, no seam impedance, extremely deep long-horizon meta-reasoning | Very high compute cost, higher latency, non-inflatable parameter count | Frontier science, cross-disciplinary system design, hard agentic closed-loop decisions |
Conclusion: Trading margin for intelligence
In the industrial carnival of lowering per-token cost, trillion-parameter narratives, and benchmark saturation, MoE is an enormously successful engineering arbitrage that spread LLM applications at minimal cost. But Fable stands as a sobering engineering parable:
> True general intelligence cannot be assembled by stitching broken expert subspaces; it requires the purest compute, irrigating every parameter to crystal density in the flood of the highest-quality corpus.
When a system is stripped of the escape hatch of sparse routing and forced to confront the physical cost of full matrix multiplication, the globally transparent general representation it deposits (the G-factor) forms a deep barrier that no single-benchmark specialist can cross. On the road toward AGI, trade margin for intelligence—simplicity ultimately prevails.
--- *Note: The "Claude Dense hypothesis" and architecture of the referenced "Fable" model are industry speculation discussed in the original post, not confirmed facts.*