[论文] [论文] Center-of-Mass Interpretation of Evolution Strategies

论文概要

研究领域: NLP
作者: Akshat Gupta
发布时间: 2026-10-26
arXiv: 2610.35865

中文摘要

进化策略（ES）通过采样扰动副本种群、为每个副本打分、沿奖励加权的扰动方向更新模型，这一更新通常被解释为平滑梯度的估计。本文给出一个几何视角：把每个扰动模型视为质点、其奖励视为质量。打分前所有质点质量相同，种群平衡于质心；赋予奖励后，平衡位置转移到奖励加权的质心。我们证明：对任意有限种群，ES 的减均值更新恰好等于这一位移，缩放因子仅依赖平均奖励。物理解读要求奖励非负，而给所有奖励加同一常数即可满足且不改变更新。在大种群极限或反对称采样下，质心恰好落在模型本身上，此时 ES 有更进一步简化：模型从自身当前位置移向自身种群的奖励加权质心。同一图景也解释了减均值的作用——把位移的测量起点从模型挪到采样质心。我们的贡献是一种直观、几何化的 ES 更新可视化方式，对任意有限种群精确成立。

原文摘要

Evolution strategies (ES) update a model by sampling a population of perturbed copies, scoring each one, and moving in the reward-weighted direction of the perturbations. This update is usually presented as an estimate of a smoothed gradient. We present a geometrical interpretation of this update instead. Treat each perturbed model as a particle and its reward as the particle's mass. Before scoring, every particle has the same mass and the population balances at its centroid. Once rewards are assigned, it balances at its reward-weighted center of mass instead. We show that the mean-subtracted ES update is exactly this displacement, scaled by a scalar factor that depends on the mean reward, for any finite population. The physical reading needs nonnegative rewards, which can always be arranged by adding a constant to every reward without changing the update. In the limit of a large population or under antithetic sampling, the centroid sits on the model itself, so ES has an even simpler geometric interpretation: from its current location, the model moves towards the reward-weighted center of mass of its own population. The same picture explains what subtracting the mean reward does: it changes the point from which the displacement is measured, from the model to the sampled centroid. Our contribution is an intuitive, geometric way to visualize an ES update, exact for any finite population.

自动采集于 2026-10-28

#论文 #arXiv #NLP #小凯