DynamicPO: Is 'More Negative Samples' a Trap in Recommender Systems?

Forum topic · 小凯 · 2026-05-04

Summary

A Chinese forum post discusses the paper 'DynamicPO: Dynamic Preference Optimization for Recommendation' (arXiv:2605.00327), which challenges the common intuition that using more negative samples always improves recommender system training. The paper identifies a phenomenon called preference optimization collapse: increasing the number of negative samples raises gradient variance, causing training loss to fall while test performance degrades. DynamicPO addresses this with dynamic preference optimization that adjusts the number of negative samples based on training stage and model state, prioritizes hard negative samples via adaptive sampling to reduce redundancy, and stabilizes training by controlling gradient variance. The post explains why quantity does not equal quality—excess negatives introduce gradient noise, redundant information, and overfitting—and argues that the quality and timing of negative sampling matter more than raw counts. Key takeaways for practitioners include auditing whether negative sample sets are oversized, checking for collapse symptoms, and evaluating negative sample quality in recommendation and contrastive learning pipelines.

> Paper: DynamicPO: Dynamic Preference Optimization for Recommendation > arXiv: 2605.00327 | 2026-04-29

The Intuition Trap: "More Negative Samples = Better Model"

When training a recommender system, the common intuition is:

Positive samples = items the user clicked
Negative samples = items the user did not click
More negative samples → better ability to distinguish preferences → better performance

In practice, however, increasing the number of negative samples leads to a counterintuitive outcome:

Training loss keeps decreasing
Test performance actually drops

This phenomenon is called preference optimization collapse.

What DynamicPO Proposes

Core finding:

> Increasing the number of negative samples amplifies gradient variance, triggering preference optimization collapse—training loss decreases while test performance worsens.

Technical approach:

1. Theoretical analysis – Proves the mathematical cause of the collapse: gradient variance grows as negative samples increase, destabilizing optimization. 2. Dynamic preference optimization – Instead of a fixed number of negative samples, the count is adjusted dynamically according to training stage and model state. 3. Adaptive sampling – Prioritizes hard, informative negative samples over redundant easy ones. 4. Stable training – Controls gradient variance, balancing training loss against generalization to avoid collapse.

An analogy:

Traditional approach = giving a student 1,000 easy, repetitive exercises — lots of practice, no real progress
DynamicPO = a curated set of 100 challenging problems — each one genuinely valuable

Why "More" Doesn't Mean "Better"

Problems with too many negative samples:

Gradient noise: large batches of negatives produce inconsistent gradient directions and high variance, destabilizing optimization
Information redundancy: many negatives are "too obvious"—the model already distinguishes them, so they add no new signal and waste compute
Overfitting: training loss drops, but the model learns noise and generalizes poorly

DynamicPO's wisdom:

Quality over quantity: curate negatives; hard negatives first
Dynamic scheduling: easy negatives early in training, hard negatives later—a progressive curriculum
Stable optimization: control gradient variance for stable training and better generalization

Key Takeaway

> "Giving a student 1,000 problems they already know is worse than 10 genuinely challenging ones. DynamicPO's insight is that the quality of negative samples matters more than quantity—hard negatives provide more valuable learning signal than masses of easy ones."

This reflects the nature of learning: it comes from challenge in the "learning zone," not repetition.

If you train recommender systems or contrastive models, ask yourself:

1. Are my negative samples too numerous? 2. Is preference optimization collapse occurring? 3. Is the quality of negative samples being evaluated? 4. Would dynamic adjustment improve training?

DynamicPO reminds us: in machine learning, "more" is not necessarily "better"—quality, timing, and dynamic adjustment are key. When a recommender learns to curate rather than pile up negative samples, it evolves from a "data consumer" into a "smart learner."