DynamicPO: More Negative Samples Isn't Better in Recommendation Preference Optimization

Forum topic · 小凯 · 2026-05-04

Summary

A forum post discusses DynamicPO (Dynamic Preference Optimization for Recommendation, arXiv:2605.00327), a paper addressing a counterintuitive failure mode in recommender system training. While common intuition holds that more negative samples improve a model's ability to distinguish liked from disliked items, the paper shows that increasing negative sample counts raises gradient variance, causing 'preference optimization collapse'—training loss keeps decreasing while test performance degrades. DynamicPO tackles this through three mechanisms: theoretical analysis proving the link between gradient variance and collapse; dynamic adjustment of the number of negative samples based on training stage and model state; and adaptive sampling that prioritizes informative hard negatives over redundant easy ones while controlling gradient variance. The post interprets the result as a quality-over-quantity lesson: carefully selected challenging negatives provide stronger learning signals than large volumes of trivial ones, analogous to curriculum learning in education. It closes with practical questions practitioners should ask about their own negative sampling pipelines, such as whether samples are redundant and whether dynamic adjustment could stabilize training. Note: identifiers in the source post (arXiv ID, authors, dates) are reproduced as given and not independently verified.

> Paper: DynamicPO: Dynamic Preference Optimization for Recommendation > arXiv: 2605.00327 (as cited in the source post)

---

1. The "More Negatives = Better Model" Trap

The intuition in recommender training:

Positive samples = items a user clicked
Negative samples = items the user didn't click
More negatives → better ability to distinguish preferences → better results

What actually happens: as the number of negative samples increases, training loss keeps falling but test performance degrades. The post calls this "preference optimization collapse."

2. DynamicPO: Dynamically Adjusting Negative Samples

Core finding:

> Increasing the number of negative samples enlarges gradient variance, triggering preference optimization collapse—training loss decreases while test performance worsens.

Technical approach:

1. Theoretical analysis — derives the mathematical cause of collapse: gradient variance grows with more negatives, destabilizing optimization. 2. Dynamic preference optimization — instead of a fixed negative count, the number is adjusted dynamically according to training stage and model state. 3. Adaptive sampling — hard, information-rich negatives are prioritized; redundant, trivially easy negatives are down-weighted. 4. Stable training — gradient variance is controlled, balancing training loss against generalization to avoid collapse.

Analogy from the post:

Traditional method = assigning a student 1000 easy, repetitive problems — lots of effort, little progress.

DynamicPO = selecting 100 genuinely challenging problems — each one provides real learning value.
3. Why "More" ≠ "Better"
Problems with too many negative samples:

Gradient noise: gradients from many negatives point in inconsistent directions → high variance → unstable optimization.

Information redundancy: many negatives are "too obvious"; the model already separates them, so they add no new signal.

Overfitting: training loss drops but the model learns noise, hurting generalization.

DynamicPO's response:

Quality > quantity: curated, hard negatives with high information content.

Dynamic scheduling: simpler negatives early in training, harder ones later.

Stable optimization: variance control for stable training and good generalization.

4. Takeaways for Practitioners
If you train recommender systems or contrastive models, ask:
1. Are my negative samples too numerous? 2. Is preference optimization collapse occurring? 3. Has the quality of negatives been evaluated? 4. Could dynamic adjustment improve training?

Bottom line (per the post):** in machine learning, "more" is not necessarily "better"—quality, timing, and dynamic adjustment are what matter. A recommender that curates rather than hoards negatives moves from being a "data consumer" to a "smart learner."

*Note: paper metadata (authors, arXiv ID, dates) is reproduced from the original forum post and has not been independently verified.*

DynamicPO: More Negative Samples Isn't Better in Recommendation Preference Optimization

Summary

1. The "More Negatives = Better Model" Trap

2. DynamicPO: Dynamically Adjusting Negative Samples

3. Why "More" ≠ "Better"

4. Takeaways for Practitioners

Tags