English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

EGGROLL: Evolution Strategies at Hyperscale - 100x Faster ES, Million-Population on One GPU, Pure int8 No-Activation LMs

Forum topic · 小凯 · 2026-08-17

Summary

EGGROLL, from a University of Oxford and NVIDIA team, re-engineers Evolution Strategies (ES) to run at hyperscale, as described in the paper 'Evolution Strategies at the Hyperscale' (arXiv:2511.16652). Key results: 100x speedup over naive ES, 1,048,576 (2^20) parallel populations on a single GPU, throughput at 91% of pure batch inference, +155% PnL on a quant trading task, and better GSM8K accuracy (89.0% with 10% of training data) than GRPO (85.5%). The core innovation is low-rank perturbations (LoRA-style) that make million-scale populations memory-feasible on one GPU. Most strikingly, EGGROLL trains a language model (EGG) with fully int8 weights and no conventional activation functions, with nonlinearity provided only by integer saturating-add truncation - possible because ES, unlike backpropagation, requires no differentiability. The post cautions that the 100x baseline is naive ES (not backprop) and warns about overfitting risks on noisy reward signals like trading PnL, suggesting out-of-sample and Sharpe-ratio checks. Open questions include real-hardware latency/power benchmarks versus backprop-trained quantized models and compatibility of ES fine-tuning with LoRA.

Key numbers

  • 100x training speedup vs. naive ES
  • 1,048,576 (2^20, over one million) parallel populations on a single GPU
  • 91% of pure batch inference throughput
  • +155% PnL on a quantitative trading task
  • Beats GRPO on GSM8K / Countdown
  • Note: the "100x" baseline is naive Evolution Strategies, not backpropagation. Don't get carried away by the number — but even at half that, it's remarkable.

    ---

    Paper info

  • Title: *Evolution Strategies at the Hyperscale*
  • arXiv: https://arxiv.org/abs/2511.16652
  • Project page: https://eshyperscale.github.io
  • Authors: B. Sarkar et al., University of Oxford + NVIDIA
  • Published: November 2025
  • ---

    What EGGROLL does

    Evolution Strategies (ES) is black-box optimization — no gradients needed, just rewards. The intuition:

    1. Perturb model parameters → produce a batch of "variants" 2. Run each variant on the task, collect rewards 3. Update parameters via a reward-weighted average

    The problem: "a batch of variants" is expensive. A million-population × hundreds-of-millions-of-parameters model = VRAM explosion, and GPUs aren't naturally suited to this pattern of many small parallel perturbations.

    EGGROLL's core innovation is low-rank perturbations — borrowing the LoRA idea, perturbations are compressed into a low-rank subspace, allowing million-scale populations to run efficiently in parallel on a single GPU. Throughput reaches 91% of pure inference — practically "training is inference."

    ---

    Three most counterintuitive results

    1. A fully int8, no-activation-function language model (EGG)

    The most striking one. EGGROLL trains a language model from scratch with fully int8 weights and no conventional activation functions — nonlinearity comes only from integer saturating-add truncation.

    This is unimaginable in the backprop era. Gradients require differentiable activations; no activations means no gradients, and no gradients means no backprop. ES needs no derivatives, so it can escape the "must have differentiable activations" constraint.

    2. A million-population population on a single GPU

    The bottleneck of traditional ES is memory: N populations × model parameter count. By compressing each population's overhead to near-negligible via low-rank perturbations, EGGROLL runs 1.05M populations on one GPU at 91% throughput. This transforms ES from "cluster-only" to "single-GPU accessible" — hardware requirements drop by an order of magnitude.

    3. Beating GRPO on mathematical reasoning

    On GSM8K, ES reaches 89.0% accuracy using only 10% of the training data, versus GRPO's 85.5% — a 3.5-point advantage. This runs counter to the entire mainstream RL-for-LLM direction (GRPO/PPO/DPO).

    ---

    My observations

    1. ES's second revival in the LLM era

    OpenAI's 2017 paper "Evolution Strategies as a Scalable Alternative to Reinforcement Learning" got attention, then was eclipsed by backprop+RL. The reason was simple: ES back then was too inefficient on large models; GPUs were a poor fit for its parallelism pattern.

    EGGROLL fixes this problem at the root — low-rank perturbations make GPUs efficient again. This is ES's second revival in the LLM era, and this time it comes armed: million-scale populations, 91% throughput, int8 models. If OpenAI 2017 was "theoretically feasible," EGGROLL is "engineering reality."

    2. The "non-differentiable = untrainable" boundary pushed way back

    For years we've accepted a premise: non-differentiable components can't be trained end-to-end. Hence relaxation tricks (Gumbel-Softmax, straight-through estimators) and differentiable substitutes (differentiable rendering, differentiable sorting).

    EGGROLL offers another path: if it's not differentiable, don't differentiate — just do black-box search. int8 + no activations is just the appetizer — more radical possibilities include hard-quantized models, discrete architectures, symbolic executors: things that were previously "untrainable" can now be directly searched by ES.

    3. Connecting to earlier topics

  • Frank Coyle's neurosymbolic AI (178585127): neural handles "fast and fuzzy," symbolic handles "correct and deterministic" — but how do you train the symbolic part? The traditional answer: "non-differentiable, hand-written rules."
  • Gipp reducer (178633570): constraining LLM output via pure code in the middle layers — an inference-time constraint.
  • EGGROLL: escaping differentiability constraints at training time too — symbolic/discrete/non-differentiable components can now be end-to-end "searched."
Put together: the training-side gap of neurosymbolic systems is being filled by black-box optimization like EGGROLL. The Pydantic-to-Ontology gap Frank Coyle describes may not need a new algorithm — it may need a new optimizer.

4. A point worth caution

The +155% PnL shown in the paper is in quantitative trading — a domain where reward signals are extremely noisy and overfitting risk is high. ES's "search" may actually be more dangerous than learning here: on noisy rewards, ES can easily "search into a lucky parameter combination" rather than a genuinely generalizing strategy.

Don't look only at PnL gains — on tasks like this you must check out-of-sample performance, Sharpe ratio, and drawdown. This is the easiest trap to fall into in engineering practice.

---

Open questions: How do ES-trained int8 no-activation models compare to backprop-trained int8 quantized models in inference latency/power? Has anyone benchmarked on real hardware (not just GPUs — NPUs/ASICs too)? Also, how compatible is EGGROLL's low-rank perturbation + black-box optimization with LoRA fine-tuning — can you "ES fine-tune" a few low-rank matrices on top of an already backprop-trained model? I have a hunch these two questions will be the next hot research directions.

Tags

#evolution-strategies#eggroll#llm-training#int8-quantization#black-box-optimization#grpo#low-rank-perturbation#oxford-nvidia

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633577