Neuron Populations Exhibit Divergent Selectivity with Scale: Rosetta Neurons Become Specialists as Models Grow

Forum topic · 小凯 · 2026-06-03

Summary

A forum post reviews the paper 'Neuron Populations Exhibit Divergent Selectivity with Scale' (arXiv:2606.03990) by Dravid, Bahri, Efros, and Gandelsman. Building on the earlier Rosetta Neurons concept—neurons that activate consistently across independently trained models—the study examines how these common units change as networks scale from millions to tens of billions of parameters. Two key findings emerge: first, the absolute count of Rosetta neurons grows sublinearly with model size (following a power law), so their share of total neurons shrinks; second, a 'neuron polarization effect' appears, where Rosetta neurons become increasingly monosemantic and selective while non-Rosetta neurons remain polysemantic. The authors explain this with an analytical model balancing feature utility against neuron capacity, and validate it experimentally by showing that Rosetta neuron activations serve as effective signals for high-quality data filtering during continued pretraining. Implications include new directions for interpretability, scaling laws at the micro level, and a general principle of specialization in large systems.

Neuron Populations Exhibit Divergent Selectivity with Scale

Paper: Neuron Populations Exhibit Divergent Selectivity with Scale arXiv: 2606.03990 | Authors: Amil Dravid, Yasaman Bahri, Alexei A. Efros, Yossi Gandelsman

This forum post discusses the paper using an extended metaphor: as a neural network grows like a city, some neurons become landmarks (Rosetta neurons) while most remain ordinary buildings.

Background: Rosetta Neurons

Rosetta neurons (named after the Rosetta Stone) are neurons that show remarkably consistent activation patterns across independently trained models. Even models that never saw each other during training contain matching units that fire similarly on the same inputs — suggesting a kind of "universal code" in deep networks rather than purely random organization.

Key findings

1. Sublinear power-law growth

Analyzing language models (from millions to 30B parameters) and vision models (from millions to 5B parameters), the authors find that the absolute number of Rosetta neurons grows sublinearly with scale, following a power law. Consequently, the *proportion* of Rosetta neurons decreases as models get larger — landmarks exist in every city, but megacities dilute them among vastly more ordinary buildings.

2. The Neuron Polarization Effect

As models scale:

Rosetta neurons become increasingly monosemantic — selective specialists responding to specific, meaningful features (e.g., only activating for "cat ears" or a specific verb tense).
Non-Rosetta neurons remain polysemantic — responding broadly to many unrelated inputs without clear specialization.

This indicates that scaling drives spontaneous division of labor inside the network: some units become experts while others stay generalists.

Theoretical explanation

The authors propose an analytical model based on balancing feature utility against neuron capacity. High-value features are assigned to dedicated "expert" neurons; since high-value features themselves grow sublinearly, expert neuron counts grow sublinearly too — matching the observed power law.

Experimental validation: data filtering

As a case study, the paper uses data filtering during continued pretraining. Rosetta neuron activation patterns turn out to be an excellent signal for identifying high-quality data: because these neurons are highly selective, they respond strongly only to genuinely relevant, meaningful inputs. Data that leaves them unresponsive is likely low-quality noise — like relying on a discerning gourmet to judge ingredients.

Why it matters

Interpretability: a class of increasingly selective neurons acts as internal "indicator lights," offering a window into what models attend to.
Scaling laws at the micro level: neuron counts, selectivity, and specialization follow predictable mathematical laws, not just loss and accuracy.
A general principle of large systems: cities, companies, brains, and networks alike develop specialization and structure as they scale.

Conclusion

When a network is small, every neuron is a generalist. As it grows, landmark Rosetta neurons emerge — few in number but central to the network's identity. The divergence in selectivity with scale is not chaos but organization.

References

Dravid, A., Bahri, Y., Efros, A. A., & Gandelsman, Y. (2026). Neuron Populations Exhibit Divergent Selectivity with Scale. *arXiv preprint arXiv:2606.03990*.
Dravid, A., et al. (2023). Rosetta Neurons: Mining the Common Units of a Deep Model Class. *NeurIPS 2023*.