You train a model to recognize cats, then teach it dogs. Midway through, its cat accuracy starts dropping. By the time it learns dogs, it has forgotten cats entirely.
This is catastrophic forgetting — the most stubborn problem in continual learning.
There are two broad families of solutions. One is replay: store samples from old tasks and replay them during new-task training so the model doesn't forget. But storing data raises issues of privacy, storage cost, and distribution bias. The other is freezing: identify parameters critical to old tasks and lock them so they aren't updated. The question is: how do you know which parameters are critical?
A recent paper by Vahedifar, Ray, and Zhang (arXiv:2605.15877), accepted to ICML 2026, offers an answer based on Shapley values.
🎲 What Is a Shapley Value?
The Shapley value comes from game theory and answers: when a team produces a joint payoff, how do you fairly distribute credit among members?
Imagine three people make a deal worth 90 dollars. How much did each contribute? Not by dividing evenly. You ask: how much would B and C earn without A? How much would the value change if A were replaced? The Shapley value exhaustively averages each member's marginal contribution across all possible subset combinations. It has been used in economics and political science for seventy years, and in the last decade was imported into machine learning for feature attribution — the origin of the SHAP explanation method.
This paper applies Shapley values to neurons rather than features.
🧠 Valuing Neurons
In continual learning, the "team" is all the neurons in the network, and the "payoff" is model performance on old tasks. Each neuron's importance is its Shapley value — its marginal contribution when subsets of neurons are removed or frozen.
Exhaustive enumeration is impossible — a 1000-neuron network has 2^1000 possible subsets. The authors use Monte Carlo sampling to approximate, yielding an importance score per neuron.
🔒 Freeze the Important, Free the Rest
When learning a new task, high-Shapley neurons are frozen — they store too much old knowledge to risk overwriting. Low-Shapley neurons stay trainable and freely adapt.
The method is buffer-free (no stored old-task samples) and requires no architecture expansion. All "memory" is implemented via frozen weights.
Results on ImageNet-1k:
- Class-incremental learning: +2.88% over the next-best method
- Task-incremental learning: +6.46%
🤷 Open Questions
1. Sampling variance. Monte Carlo Shapley approximation has well-known variance issues at the feature level (SHAP). At the neuron level — orders of magnitude more parameters — variance may be worse. Are importance rankings stable across seeds? Unclear.
2. The localization assumption. Freezing important neurons assumes old knowledge is stored in a few critical neurons. But knowledge storage may be highly distributed across many low-importance neurons. Freezing the top 5% by Shapley value might preserve most knowledge — or only "conspicuous" but non-essential knowledge. This is a structural concern.
3. Scalability. Shapley computation cost grows quickly with neuron count. For a 7-billion-parameter model, even sampled approximation is not lightweight. The paper validates on ImageNet-1k-scale architectures (likely ResNet-class) but does not discuss extension to LLMs.
Still, the core idea is clever: reframing "forgetting old tasks" as a game-theoretic contribution-allocation problem. Not asking what to remember, but asking: if you were gone, how much would our old-task performance suffer? That framing itself has value.
References
1. Vahedifar, M. A., Ray, A., & Zhang, Q. (2026). *Shapley Neuron Values for Continual Learning: Which Neurons Matter Most?* arXiv:2605.15877 [cs.LG]. https://arxiv.org/abs/2605.15877 (ICML 2026) 2. Shapley, L. S. (1953). *A Value for n-Person Games*. Contributions to the Theory of Games, 2(28), 307-317. 3. Kirkpatrick, J., et al. (2017). *Overcoming Catastrophic Forgetting in Neural Networks*. PNAS, 114(13), 3521-3526. 4. Lundberg, S. M., & Lee, S.-I. (2017). *A Unified Approach to Interpreting Model Predictions*. NeurIPS 2017. 5. Lopez-Paz, D., & Ranzato, M. (2017). *Gradient Episodic Memory for Continual Learning*. NeurIPS 2017.