Learned, Then Lost: Measuring a Single Training Example's Counterfactual Effect in GPT-2 Pre-training

Forum topic · 小凯 · 2026-08-21

Summary

A new arXiv paper (2608.19168) by Zachary Speck and Asa Shepard measures, rather than estimates, the contribution of a single training example to a finished model. The authors trained 32 GPT-2 124M models from scratch on OpenWebText across four conditions and eight seeds, replacing one row of a 256-row batch at step 200 of 9,536 with a fixed context injection carrying a 194-token passage (corpus-attested prose, gradient-matched fabricated prose, random keyboard characters, or no injection). The injected passage was learned from a single exposure—improving cross-entropy by 0.039–0.044 nats across all eight seeds (p<10^-4)—then decayed to undetectability by the final step. Geometric measurements taken after decay showed weight displacement reaching 44.1% of seed-to-seed distance while interpolation loss barriers reached only 3.0%, indicating the injection relocates the model within its loss basin without moving it out. Held-out cross-entropy and pre-registered loss barriers showed no significant differences.

Paper Overview

Field: Machine Learning Authors: Zachary Speck, Asa Shepard Published: 2026-08-19 arXiv: 2608.19168

Abstract

A single training example's contribution to a finished model is normally estimated rather than measured, because measuring it takes two expensive full pre-training runs that differ in one row of one batch. We ran that counterfactual 24 times at a small scale. We trained 32 GPT-2 models at 124M parameters from scratch on OpenWebText, over four conditions and eight seeds. At step 200 of 9,536, at peak learning rate, we replaced one row of a 256-row batch with a fixed context injection carrying a 194-token passage.

The three injected conditions are:

1. Fluent prose with a corpus-attested subject 2. Fluent prose with a fabricated subject matched to it within 0.14% on full-batch gradient delta 3. Random keyboard characters

The fourth condition is an uninjected twin.

Key Findings

The passage is learned from one exposure, then decays: 50 steps after injection, arms that saw the passage predict it better than arms that did not, with differences of 0.039 and 0.044 nats of cross-entropy, consistent across all eight seeds, p < 10^-4.
The effect vanishes by the end: At the final step, no difference was detected for either passage (p = 0.25 and p = 0.71) against minimum detectable effects of 0.025 and 0.079 nats; no difference between the two passages either (p = 0.54).
All geometric measurements were taken after decay: The pre-registered interpolation loss barrier contrast was +0.0068, p = 0.509, against a minimum detectable effect of 0.032 barrier units. Held-out cross-entropy difference was -0.00044, p = 0.310.
Per-layer centered kernel alignment (CKA) did not detectably separate for any condition at any layer.
Weights move, basins don't: Weight displacement reached 44.1% of seed-to-seed Euclidean distance and was 92% stable before the training midpoint, while the barrier reached only 3.0% of the seed-to-seed barrier. These two figures are roughly a factor of 15 apart, which is a lower bound.

Conclusion

The injection relocates the model *within* a loss basin, without moving it out of the basin: a single example can leave a large, measurable trace in the weights even after its behavioral effect on the injected passage has fully decayed.