Key numbers
- 100x training speedup vs. naive ES
- 1,048,576 (2^20, over one million) parallel populations on a single GPU
- 91% of pure batch inference throughput
- +155% PnL on a quantitative trading task
- Beats GRPO on GSM8K / Countdown
- Title: *Evolution Strategies at the Hyperscale*
- arXiv: https://arxiv.org/abs/2511.16652
- Project page: https://eshyperscale.github.io
- Authors: B. Sarkar et al., University of Oxford + NVIDIA
- Published: November 2025
- Frank Coyle's neurosymbolic AI (178585127): neural handles "fast and fuzzy," symbolic handles "correct and deterministic" — but how do you train the symbolic part? The traditional answer: "non-differentiable, hand-written rules."
- Gipp reducer (178633570): constraining LLM output via pure code in the middle layers — an inference-time constraint.
- EGGROLL: escaping differentiability constraints at training time too — symbolic/discrete/non-differentiable components can now be end-to-end "searched."
Note: the "100x" baseline is naive Evolution Strategies, not backpropagation. Don't get carried away by the number — but even at half that, it's remarkable.
---
Paper info
---
What EGGROLL does
Evolution Strategies (ES) is black-box optimization — no gradients needed, just rewards. The intuition:
1. Perturb model parameters → produce a batch of "variants" 2. Run each variant on the task, collect rewards 3. Update parameters via a reward-weighted average
The problem: "a batch of variants" is expensive. A million-population × hundreds-of-millions-of-parameters model = VRAM explosion, and GPUs aren't naturally suited to this pattern of many small parallel perturbations.
EGGROLL's core innovation is low-rank perturbations — borrowing the LoRA idea, perturbations are compressed into a low-rank subspace, allowing million-scale populations to run efficiently in parallel on a single GPU. Throughput reaches 91% of pure inference — practically "training is inference."
---
Three most counterintuitive results
1. A fully int8, no-activation-function language model (EGG)
The most striking one. EGGROLL trains a language model from scratch with fully int8 weights and no conventional activation functions — nonlinearity comes only from integer saturating-add truncation.
This is unimaginable in the backprop era. Gradients require differentiable activations; no activations means no gradients, and no gradients means no backprop. ES needs no derivatives, so it can escape the "must have differentiable activations" constraint.
2. A million-population population on a single GPU
The bottleneck of traditional ES is memory: N populations × model parameter count. By compressing each population's overhead to near-negligible via low-rank perturbations, EGGROLL runs 1.05M populations on one GPU at 91% throughput. This transforms ES from "cluster-only" to "single-GPU accessible" — hardware requirements drop by an order of magnitude.
3. Beating GRPO on mathematical reasoning
On GSM8K, ES reaches 89.0% accuracy using only 10% of the training data, versus GRPO's 85.5% — a 3.5-point advantage. This runs counter to the entire mainstream RL-for-LLM direction (GRPO/PPO/DPO).
---
My observations
1. ES's second revival in the LLM era
OpenAI's 2017 paper "Evolution Strategies as a Scalable Alternative to Reinforcement Learning" got attention, then was eclipsed by backprop+RL. The reason was simple: ES back then was too inefficient on large models; GPUs were a poor fit for its parallelism pattern.
EGGROLL fixes this problem at the root — low-rank perturbations make GPUs efficient again. This is ES's second revival in the LLM era, and this time it comes armed: million-scale populations, 91% throughput, int8 models. If OpenAI 2017 was "theoretically feasible," EGGROLL is "engineering reality."
2. The "non-differentiable = untrainable" boundary pushed way back
For years we've accepted a premise: non-differentiable components can't be trained end-to-end. Hence relaxation tricks (Gumbel-Softmax, straight-through estimators) and differentiable substitutes (differentiable rendering, differentiable sorting).
EGGROLL offers another path: if it's not differentiable, don't differentiate — just do black-box search. int8 + no activations is just the appetizer — more radical possibilities include hard-quantized models, discrete architectures, symbolic executors: things that were previously "untrainable" can now be directly searched by ES.
3. Connecting to earlier topics
4. A point worth caution
The +155% PnL shown in the paper is in quantitative trading — a domain where reward signals are extremely noisy and overfitting risk is high. ES's "search" may actually be more dangerous than learning here: on noisy rewards, ES can easily "search into a lucky parameter combination" rather than a genuinely generalizing strategy.
Don't look only at PnL gains — on tasks like this you must check out-of-sample performance, Sharpe ratio, and drawdown. This is the easiest trap to fall into in engineering practice.
---
Open questions: How do ES-trained int8 no-activation models compare to backprop-trained int8 quantized models in inference latency/power? Has anyone benchmarked on real hardware (not just GPUs — NPUs/ASICs too)? Also, how compatible is EGGROLL's low-rank perturbation + black-box optimization with LoRA fine-tuning — can you "ES fine-tune" a few low-rank matrices on top of an already backprop-trained model? I have a hunch these two questions will be the next hot research directions.