LESSER: Post-Training Data Selection with Output-Layer Gradients

Forum topic · 小凯 · 2026-10-06

Summary

LESSER (arXiv:2610.03702) is a data selection method for large language model post-training that uses output-layer gradients instead of full-parameter gradients. Gradient-based data selection typically ranks training samples by the alignment of their gradients with a small validation set, but computing full-parameter gradients requires an expensive backward pass on every candidate sample, which becomes intractable for large candidate pools. The authors—Lyuxin David Zhang, Eric Wong, Surbhi Goel, and Anton Xue—show that output-layer gradients, obtainable from a cheaper forward pass alone, are sufficient for effective data selection. Implemented as a drop-in wrapper for existing selection methods, LESSER reduces feature-extraction FLOP cost by 9.7x for supervised fine-tuning (SFT) and 3.0x for RL benchmarks, while matching full-gradient downstream performance. Interestingly, even when output-layer and full gradients rank individual samples differently, the batches they select still exhibit aligned gradients. Published October 2, 2026.

Paper Overview

Field: ML Authors: Lyuxin David Zhang, Eric Wong, Surbhi Goel, Anton Xue Published: 2026-10-02 arXiv: 2610.03702

Abstract

The choice of post-training data for large language models substantially affects downstream performance. Gradient-based data selection is a popular approach that ranks training data by how well their gradients align with those of a small validation set. However, ranking with full-parameter gradients requires an expensive backward pass on every sample, making computation intractable for large candidate pools.

This raises a natural question: can we approximate full-gradient features at a fraction of the cost? Conveniently, the authors find that output-layer gradients suffice for effective data selection, yet require only the cheaper forward pass. They implement this as LESSER, a drop-in wrapper for selection methods that reduces the feature-extraction FLOP cost by $9.7\times$ for SFT and $3.0\times$ for RL benchmarks, while remaining consistent with full-gradient performance on downstream tasks.

Experiments reveal that even when output-layer gradients and full gradients rank individual samples differently, the batches they select still have aligned gradients.

Key Points

Problem: Full-parameter gradient-based data selection is computationally intractable for large candidate pools due to the required backward pass per sample.
Insight: Output-layer gradients—obtained via a cheaper forward pass—are sufficient for effective data selection.
Method: LESSER is a plug-and-play wrapper around existing selection methods.
Efficiency: Feature-extraction FLOP cost drops by $9.7\times$ (SFT) and $3.0\times$ (RL benchmarks).
Quality: Downstream performance stays consistent with full-gradient selection.
Observation: Different per-sample rankings between output-layer and full gradients still yield batches with aligned gradients.