Neuron Populations Exhibit Divergent Selectivity with Scale: Landmark Neurons in Growing AI Models

Forum topic · 小凯 · 2026-06-03

Summary

This forum post reviews the paper "Neuron Populations Exhibit Divergent Selectivity with Scale" (arXiv:2606.03990) by Dravid, Bahri, Efros, and Gandelsman. Building on the concept of Rosetta Neurons—units that activate consistently across independently trained models—the study finds that as language models (up to 30B parameters) and vision models (up to 5B parameters) scale, the number of Rosetta neurons grows sublinearly following a power law, so their share of total neurons shrinks even as their absolute count rises. The paper further identifies a "neuron polarization effect": Rosetta neurons become increasingly monosemantic and selective, while non-Rosetta neurons remain polysemantic, suggesting networks self-organize into specialists and generalists as they grow. The authors explain this via an analytical model balancing feature utility against neuron capacity, and demonstrate a practical application: Rosetta neuron activation patterns serve as effective signals for filtering high-quality training data during continued pretraining. The findings extend scaling laws to the micro level and offer new hope for interpretability research.

*English translation of a zhichai.net forum post reviewing a research paper.*

Paper: Neuron Populations Exhibit Divergent Selectivity with Scale — When AI Grows Up, Some Cells Become Landmarks, Others Passersby

Neuron Populations Exhibit Divergent Selectivity with Scale arXiv: 2606.03990 | Authors: Amil Dravid, Yasaman Bahri, Alexei A. Efros, Yossi Gandelsman

Introduction: The Growth Code of a City

Imagine standing on a hilltop overlooking a city. When it was founded, there were only a few buildings and roads, and everyone knew each other. But as the city expanded, skyscrapers rose and neighborhoods grew complex. Some buildings became landmarks—the Eiffel Tower, the Empire State Building—that everyone points to. Others are just ordinary homes and offices, present but unremarked.

Neural networks grow like expanding cities. Over the past decade we have built models from millions of parameters (small villages) to billions (metropolises) to trillions (mega city-clusters). Bigger cities mean greater capability—but we rarely ask: what happens to the "residents"—the neurons—as the city grows? Do they multiply uniformly, or does differentiation emerge? Are there "landmark" neurons found in every city?

This paper answers that question.

Background: Rosetta Neurons and a Universal Code

Rosetta Neurons are named after the Rosetta Stone, which unlocked ancient Egyptian writing by providing three parallel scripts. In neural networks, Rosetta neurons are special units that show remarkable consistency across different models. Even two models trained independently—never having "seen" each other—contain neurons with nearly identical activation patterns, as if two unrelated people's brains light up in exactly the same spot when viewing the same image.

This is strange and important: it means neural networks are not purely chaotic random systems. At some deep level, a "universal code" exists.

Finding 1: A Sublinear Power Law with Scale

The first core finding concerns how the number of Rosetta neurons grows with model scale. The researchers analyzed language models (from millions to 30 billion parameters) and vision models (from millions to 5 billion parameters), and found:

The total number of Rosetta neurons grows sublinearly, following a power law, as model scale increases.

In city terms: double the population and landmarks increase too, but more slowly—perhaps only 1.5x—while ordinary buildings multiply explosively. This means the share of Rosetta neurons among all neurons actually shrinks as models grow. A small city might have 10% landmark buildings; a mega-metropolis maybe only 1%. Their absolute numbers still rise, but they get drowned in a sea of ordinary buildings.

This breaks the naive picture that scaling enlarges everything proportionally. In reality, the network's interior is differentiating—some things slow down while others accelerate.

Finding 2: The Neuron Polarization Effect

The second finding is deeper: the Neuron Polarization Effect. As models scale, the difference between Rosetta and non-Rosetta neurons grows—like social stratification in a city:

Rosetta neurons become increasingly "picky" and monosemantic, responding only to specific, meaningful features—a neuron that fires only for "cat ears" or the past tense of a verb. They are like expert connoisseurs who comment only on their specialty.
Non-Rosetta neurons stay "casual" and polysemantic, responding to many different, often unrelated inputs—ordinary residents with mild reactions to everything but no deep insight.

This divergence means networks spontaneously organize a division of labor as they scale: some neurons specialize into "experts" while others remain "generalists" handling miscellaneous tasks.

Theoretical Explanation: Why Differentiation Is Inevitable

The researchers propose an elegant analytical model based on the balance between feature utility and neuron capacity.

Picture a warehouse storing goods. Some items are critical—core components that should be placed in the most accessible spots with clear labels. These are high-value features. As the warehouse grows, a smart manager doesn't have every worker do the same chores; the most skilled workers are assigned to the most important core components and become specialists, while ordinary workers keep handling daily tasks.

The model predicts exactly the sublinear power law: specialists grow more slowly than total headcount, because the set of carefully selected high-value features they cover itself grows sublinearly.

Experimental Validation: Data Filtering as a Test of "Expertise"

The paper includes a striking experiment verifying how Rosetta neurons' "professionalism" functions during continued pretraining, using data filtering as a case study. Given a large training set with mixed-quality data, you can use the model's existing knowledge to keep samples that are genuinely useful and discard noise.

The result: Rosetta neurons' activation patterns are an excellent signal for identifying high-quality data. Because they are highly selective, they respond strongly only to relevant, meaningful inputs. If a batch of data leaves Rosetta neurons unmoved, it is likely low-quality noise—like having a veteran gourmet judge ingredients: their refined palate detects the slightest staleness that ordinary people would miss.

Why This Matters Beyond Being an Interesting Finding

First, it brings hope for interpretability. Neural networks have long been black boxes. But if a class of neurons becomes increasingly specialized with scale, they act like internal "indicator lights," letting us glimpse what the model actually attends to.

Second, it opens a new dimension for scaling laws. We usually discuss scaling laws via macro metrics—loss, accuracy, perplexity. This paper shows scaling laws also apply at the micro level: neuron counts, selectivity, and specialization follow predictable mathematical regularities.

Finally, it hints at a universal principle of organizing intelligence. Whether in human brains, neural networks, cities, or companies, systems that grow develop division of labor and specialization—almost a law of nature: growing complexity necessarily accompanies structural differentiation.

Closing: The Soul of the City

Return to the opening metaphor. When you overlook a city from a hilltop, you see not just the number of buildings but their relationships. Some buildings define the city's identity; some carry daily life; some connect people's needs.

Neural networks are the same. When small, every neuron is a generalist doing a bit of everything. But as it grows, "landmarks" emerge—the Rosetta neurons specialized for core concepts. Few in number, but deeply significant, they tell us that even in the most complex systems there is order and regularity.

As the paper's title says: neuron populations exhibit divergent selectivity with scale. This is not chaos but evolution; not dissolution but organization.

The soul of a city lies not in how many buildings it has, but in how many landmarks. The soul of a neural network may be the same.

References

Dravid, A., Bahri, Y., Efros, A. A., & Gandelsman, Y. (2026). Neuron Populations Exhibit Divergent Selectivity with Scale. *arXiv preprint arXiv:2606.03990*.
Dravid, A., et al. (2023). Rosetta Neurons: Mining the Common Units of a Deep Model Class. *NeurIPS 2023*.