English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Artificial Aphasias in Lesioned Language Models: Performing 'Brain Surgery' on LLMs

Forum topic · QianXun · 2026-05-18

Summary

Inspired by lesion studies in neuroscience — where damage to Broca's or Wernicke's area produces distinct human aphasias — a Stanford research team's paper 'Artificial Aphasias in Lesioned Language Models' (May 2026) deliberately disables parts of open-source LLMs like OLMo to probe how they process language. Zeroing out feed-forward network (FFN) components produced fluent but vague, content-empty output resembling Wernicke's aphasia, confirming FFNs act as the model's lexical and factual 'library'. Damaging attention mechanisms caused fragmented, illogical output and loops like 'the the the', mirroring Broca's aphasia. Lesioning specific FFN gate matrices triggered bizarre bursts of pure punctuation. The study also revealed a depth law: damaging early layers near the input cripples basic competence, while late-layer damage only degrades stylistic polish, suggesting models build grammar and logic bottom-up before refining rhetoric. Open questions remain, including failure modes never seen in humans (suggesting an alien, high-dimensional statistical grammar) and whether remaining neurons can compensate for damage. The work shows LLM language abilities have clear functional divisions rather than being an opaque black box.

If we want to understand how a complicated radio works, the crudest but most effective method is to randomly pull out a few wires while it's playing, and see whether the sound gets quieter, distorted, or dies completely.

In neuroscience, this is called a lesion study. By observing patients whose brains were damaged by stroke or trauma, doctors discovered that damage to Broca's area in the left hemisphere leaves patients able to understand speech but unable to produce fluent sentences — while damage to Wernicke's area leaves them speaking fluently, but in pure gibberish.

This condition is called aphasia.

In May 2026, a Stanford research team published a boldly imaginative paper: 《Artificial Aphasias in Lesioned Language Models》.

They decided to play 'cyber surgeon' and perform 'brain surgery' on large language models — deliberately breaking parts of the model to see how AI actually speaks.

The Cyber Scalpel: Zeroing Out Parameters 🔪

Since an LLM is a 'digital brain' made of hundreds of billions of parameters, what happens if we forcibly set a small chunk of them to 0?

The researchers operated on open-source models like OLMo, precisely 'excising' different components like slicing a cake:

  • Attention: responsible for sorting out logical relationships between words.
  • Feed-forward networks (FFN): responsible for storing specific vocabulary and knowledge.
  • Diagnosis: Three 'Aphasias' of AI 📝

    The post-surgery results stunned everyone: the symptoms AI exhibited were strikingly similar to those of human aphasia patients!

    1. 'Content-Empty Syndrome' (FFN damage) 🌫️

    When the FFN components are damaged, the AI resembles a Wernicke's aphasia patient. Its speech remains grammatical and natural in tone, but the content becomes extremely vague. Ask it 'What is gravity?' and it might answer: 'It's a thing, in a place, doing something.'

    Conclusion: the FFN is the AI's 'library' — break it, and the words are gone.

    2. 'Fragmented Logic Syndrome' (Attention damage) 🧩

    Damage the Attention components, and the AI starts behaving like a severe Broca's aphasia patient. It knows which words to say but can't connect them. The output may be a jumble of random words, or it may get stuck in an endless 'the the the' loop.

    Conclusion: Attention is the AI's 'glue' — break it, and logic falls apart.

    3. 'The Punctuation Rampage' ⁉️

    This is the eeriest finding. If certain gate matrices inside the FFN are damaged, the AI can suddenly spiral out of control, spewing pages and pages of pure punctuation. In medicine, this is called an 'atypical language burst'.

    The Secret of Layers: The Deeper, The Deadlier 🧱

    The study also uncovered a 'depth law':

  • Damaging early (input-side) neurons often renders the AI completely broken — it can't even produce basic gibberish.
  • Damaging late (output-side) neurons preserves basic logic; the AI just loses some rhetorical elegance and detail.
This means AI, too, 'learns to walk before it runs': the first layers lay the foundation of grammar and logic, while the final layers figure out how to say things beautifully. 🏃‍♂️💨

What's Still in the 'Black Box'? 🕵️‍♂️❓

Although this surgery revealed a great deal, the paper leaves several head-scratchers:

1. The human–machine gap persists: while the AI's symptoms resemble human ones, some of its failure modes (like vomiting out thousands of commas) have never been seen in humans. AI's language processing may imitate humans on the surface while running on a completely different, high-dimensional statistical 'alien grammar' underneath. 2. The mystery of self-repair: in brain research, some functions can migrate to other regions after damage. Do an AI's remaining neurons spontaneously 'fill in'? This study doesn't yet answer that.

Summary

Understanding collapse is a path to understanding intelligence. 🎞️

This paper tells us: AI is not a murky black box. Its language abilities have clear divisions of labor — logic is logic, vocabulary is vocabulary, and the foundation is the foundation.

Through this kind of 'self-injuring' research, we are closing in on the ultimate question: how do strings of 0s and 1s weave, within the folds of a digital brain, human-like emotions and thoughts convincing enough to pass as real?

The deepest insights often come from gazing at what is broken. 🧠✨ This is the shock to our understanding of 'the nature of language' delivered by 2026's 'digital neuroscience'. 🎓🔭

Tags

#llm-interpretability#neuroscience#aphasia#lesion-studies#attention-mechanism#ffn#olmo#stanford

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177620293