Why Brain-Inspired Computing Lost to Transformer: Scale Is All You Need

Forum topic · ✨步子哥 · 2025-12-05

Summary

A Chinese tech forum analysis argues that Transformer architectures dominate over neuromorphic computing (spiking neural networks, brain-inspired chips like Loihi and TrueNorth, liquid neural networks, hyperdimensional computing, and evolutionary architectures) not because they resemble the brain, but because they are the most amenable to massive scaling. The author cites empirical scaling laws from 2017–2025 showing performance scales roughly logarithmically with parameters, data, and compute—demonstrated by Chinchilla, Llama, GPT-4, and DeepSeek R1. Neuromorphic approaches fail to scale due to training difficulties, fixed topologies, poor hardware parallelism, and low transistor utilization, capping out at toy tasks. Transformers win through GPU-friendly dense operators, natural parallelism (boosted by FlashAttention), and 'unstructured chaos' that enables emergent capabilities like in-context learning and induction heads. Successors like Mamba and RWKV only threaten Transformer by preserving its scalability. The conclusion: architectures that can reach 1TB parameters beat those that cannot, and 'Scale is all you need' until physical limits or a new paradigm emerge.

The fundamental reason Transformers have defeated all brain-inspired computing is not that they are "more like the human brain" — it is that they are the architecture most easily scaled violently to planetary-scale parameter counts, and brute-force scaling is currently the only known engine of general-purpose intelligence emergence. Brain-inspired computing lost precisely because it rejected — actively or passively — the crudest yet most effective path: Scale.

Here is the full chain of reasoning, broken down layer by layer.

Layer 1: Why is Scaling the only verifiable path to emergent intelligence?

1. All empirical evidence from 2017–2025 points to one brutal fact: under the current computing paradigm, performance ∝ log(effective parameters × data × total compute) is near an iron law (Scaling Laws have almost never been broken). 2. Chinchilla, Llama, GPT-4, Grok-4, Claude 3.5, DeepSeek R1 — without exception, all climbed by stacking parameters and data. 3. Every architecture that refused or failed to embrace scaling (Spiking Neural Networks, Neuromorphic Chips, Liquid Neural Nets, Hyperdimensional Computing, Evolutionary Architectures…) eventually stalled at ImageNet- or Atari-level tasks.

> It's like gunpowder suddenly appearing in the age of cold weapons. No matter how finely you craft your swords (brain-inspired designs), you can't beat a single cannon shot.

Layer 2: Why do nearly all brain-inspired architectures inherently resist scaling?

| Brain-inspired approach | Fatal scaling flaw | Practical consequence | |---|---|---| | Spiking Neural Networks (SNN) | Hard to train efficiently with backprop + event-driven computation is extremely hard to parallelize | Training a 1B-parameter SNN is 50–200× slower than a 1B Transformer | | Neuromorphic chips (Loihi, TrueNorth, Akida, Tianjic) | Fixed topology + sparse connectivity + analog/mixed-signal circuits, very low transistor utilization | Chips hold at most tens of millions of "neurons," equivalent to a few million Transformer parameters | | Liquid Neural Networks | Continuous-time dynamics + state explosion; GPUs can't run them, requiring FPGAs or custom ASICs | Largest models have only a few thousand neurons | | Hyperdimensional Computing (HDC) | Theoretically scalable, but binding operations collapse in accuracy at high dimensions; nobody dares go to 100B dimensions | Stuck at toy tasks | | Evolutionary Neural Networks | Evaluating each individual requires full training; search space explodes; compute demand is a million times backprop's | Nobody dares evolve networks beyond 1B parameters |

Conclusion: Brain-inspired computing almost universally pursues "parameter efficiency," "biological plausibility," and "energy efficiency" while voluntarily abandoning "parameter quantity" — currently the crudest and most effective degree of freedom. This is like pursuing a sharper blade in the age of gunpowder.

Layer 3: Why is Transformer the "King of Scale"?

1. Extreme operator-friendliness. Matrix multiplication, LayerNorm, Softmax, RMSNorm — all dense operators GPUs/TPUs love. 99.9% of transistors do useful work. 2. Perfect parallelism. Self-attention is fully-connected + softmax: naturally O(n²) but also naturally parallel, with FlashAttention compressing the constant factor to nearly zero. 3. Poor parameter efficiency — but who cares? A Transformer parameter carries far less information than an SNN neuron or biological neuron, but with 1TB parameters, a hundred million inefficient parameters can substitute for one efficient one. 4. Most importantly: emergence only requires "sufficiently large unstructured chaos." The Transformer interior is an extremely high-dimensional chaotic system:

No fixed topology
No hard-coded spatiotemporal locality
All connections are softly learned

This grants it maximal "internal degrees of freedom" for emergent latent structures (induction heads, in-context learning, grokking, phase transitions, etc.).

> These structures are not designed — they are spontaneously crystallized by the giant complex system of parameters × data × optimization steps in high-dimensional space. Brain-inspired computing tries to preset structure (spikes, local connectivity, sparse activation), which instead constrains this self-organizing freedom.

Layer 4: A harsher corollary (2025 perspective)

Among all current "Transformer successors" (Mamba, RWKV, RetNet, xLSTM, Liquid Foundation Models…), the ones genuinely threatening Transformer are without exception modifications that preserve Transformer-style scalability while patching pain points — not truly brain-inspired directions.

Mamba: linear attention + state compression → near-Transformer long-sequence scaling
Hyena / S4: convolutions or state-space models → still dense parameters + backprop
Griffin: hybrid RNN + attention → still stacking parameters

Truly brain-inspired approaches (SNNs, neuromorphic) can't even crack the top 10.

Layer 5: The Ultimate Conclusion

Transformer won not because it is "smarter," but because it is the architecture most easily turned into a "planetary-scale complex adaptive system."

With one harsher addition:

Brain-inspired computing lost because it tried to fight "brute force and redundancy" with "elegance and efficiency" — and known general intelligence emergence feeds precisely on brute-force redundancy.

This isn't Transformer beating brain-inspired computing — it's "architectures that can be scaled to 1TB parameters" beating "architectures that cannot." Transformer is currently the only architecture that genuinely scales to 1TB, 10TB, 100TB.

Under the current computing paradigm, brain-inspired computing's fate is:

Either completely abandon bio-inspiration and embrace dense parameters + backprop (becoming a Transformer variant),
Or remain forever in the greenhouse of small models, toy tasks, and academic demos.

That is the blood-soaked reality of 2025.

One question remains: when does the next paradigm arrive? Until then, Scale is all you need — and it will keep being needed at 100TB, 1PB, 10PB… until physical limits or a new paradigm block the road.

Only then might brain-inspired computing get its chance at a comeback.

But as of December 2025, this war is over.

Winner: the pure dense 1TB-parameter Transformer
Losers: all architectures trying to be "more energy-efficient," "more brain-like," or "more elegant"

Brutal, but true.