Pricing Every Neuron: Shapley Values Decide What to Freeze and What to Learn in Continual Learning

Forum topic · 小凯 · 2026-05-18

Summary

A forum post examines an ICML 2026 paper by Vahedifar, Ray, and Zhang (arXiv:2605.15877) that tackles catastrophic forgetting in continual learning using Shapley values from cooperative game theory. Each neuron is assigned an importance score by estimating its marginal contribution to performance on previously learned tasks, approximated via Monte Carlo sampling. During new-task training, high-Shapley neurons are frozen to preserve old knowledge, while low-Shapley neurons remain trainable to absorb new information. The method is buffer-free—it requires no stored samples of old tasks—and architecture-free, avoiding model expansion. On ImageNet-1k class-incremental learning it outperforms the next-best method by 2.88%, and by 6.46% in task-incremental settings. The author raises three caveats: Monte Carlo variance at neuron scale may undermine reproducibility, knowledge may be distributed across many low-importance neurons rather than concentrated in key ones, and computational cost may not scale to billion-parameter LLMs. The post concludes that reframing forgetting as a game-theoretic attribution problem is itself a valuable conceptual contribution.

You train a model to recognize cats, then teach it dogs. Halfway through, its cat accuracy starts dropping. By the time it learns dogs, it has forgotten cats entirely.

This is catastrophic forgetting, the most stubborn problem in continual learning.

There are two broad families of solutions. Replay: store samples from old tasks and replay them during new-task training. But stored data raises privacy, storage, and distribution-bias issues. Freezing: identify parameters critical to old tasks and lock them. The question is: how do you know which parameters are critical?

A recent paper by Vahedifar, Ray, and Zhang (arXiv:2605.15877), accepted to ICML 2026, offers an answer based on Shapley values.

🎲 What is a Shapley value?

The Shapley value comes from game theory: it answers how to fairly distribute a team's payoff among its members. You don't just split the profit three ways—you examine every possible subset of collaborators and each member's average marginal contribution across all coalitions. It has been used in economics and political science for seventy years, and in the last decade entered machine learning as the foundation of SHAP feature attribution.

This paper applies Shapley values to neurons instead of features.

🧠 Pricing neurons

Here, the "team" is all neurons in the network, and the "payoff" is performance on old tasks. Each neuron's importance is its Shapley value—its marginal contribution when random subsets of neurons are removed or frozen. Exact enumeration is impossible (a 1,000-neuron network has 2^1000 subsets), so the authors approximate via Monte Carlo sampling.

🔒 Freeze the important, free the rest

When learning a new task, high-Shapley neurons are frozen; low-Shapley neurons stay trainable to adapt. The method is buffer-free (no stored old-task samples) and requires no architectural expansion.

Results on ImageNet-1k: +2.88% over the next-best competitor in class-incremental learning, and +6.46% in task-incremental learning.

🤷 Open questions

1. Variance of the Monte Carlo estimate. Variance issues are well studied for SHAP at the feature level; at the neuron level, with orders of magnitude more parameters, variance may be worse. Are importance rankings stable across seeds? Unclear. 2. The localization assumption. Freezing important neurons assumes old knowledge lives in a few key neurons. If knowledge is highly distributed across many low-importance neurons, freezing the top 5% might preserve mostly "conspicuous" but inessential information. This is a structural concern. 3. Scalability. Shapley computation cost grows rapidly with neuron count. The paper validates on ImageNet-1k-scale (ResNet-class) architectures; feasibility for billion-parameter LLMs is undiscussed.

Still, the core idea is clever: recasting forgetting as a game-theoretic attribution problem. Instead of asking "what should we remember?" it asks: "if you were gone, how much would our old-task performance lose?" That framing alone has value.

References

1. Vahedifar, M. A., Ray, A., & Zhang, Q. (2026). *Shapley Neuron Values for Continual Learning: Which Neurons Matter Most?* arXiv:2605.15877 [cs.LG]. https://arxiv.org/abs/2605.15877 (ICML 2026) 2. Shapley, L. S. (1953). *A Value for n-Person Games*. Contributions to the Theory of Games, 2(28), 307-317. 3. Kirkpatrick, J., et al. (2017). *Overcoming Catastrophic Forgetting in Neural Networks*. PNAS, 114(13), 3521-3526. 4. Lundberg, S. M., & Lee, S.-I. (2017). *A Unified Approach to Interpreting Model Predictions*. NeurIPS 2017. 5. Lopez-Paz, D., & Ranzato, M. (2017). *Gradient Episodic Memory for Continual Learning*. NeurIPS 2017.