Summary
ParetoSlider (arXiv:2604.20816) is a multi-objective reinforcement learning (MORL) framework for post-training diffusion models introduced by Shelly Golan, Michael Finkelson, and Ariel Bereslavsky. Unlike standard RL post-training that collapses multiple rewards into a fixed weighted sum, locking the model into a single trade-off point, ParetoSlider trains a single diffusion model to approximate the entire Pareto front. Continuously varying preference weights serve as a conditioning signal, allowing users to navigate optimal trade-offs between conflicting goals—such as prompt adherence versus source fidelity in image editing—at inference time, without retraining or maintaining multiple checkpoints. The framework was evaluated on three state-of-the-art flow-matching backbones (SD3.5, FluxKontext, and LTX-2), where the single preference-conditioned model matched or exceeded baselines trained for fixed reward trade-offs while uniquely enabling fine-grained control over competing generation objectives.
Overview
Field: Computer Vision
Authors: Shelly Golan, Michael Finkelson, Ariel Bereslavsky
Published: 2026-04-22
arXiv: 2604.20816
Summary
Reinforcement learning (RL) post-training has become the standard approach for aligning generative models with human preferences, but most methods rely on a single scalar reward. When multiple criteria matter, the prevailing practice of "early scalarization"—collapsing rewards into a fixed weighted sum—commits the model to a single trade-off point at training time, leaving no inference-time control over inherently conflicting objectives (e.g., prompt adherence versus source fidelity in image editing).
The authors introduce ParetoSlider, a multi-objective RL (MORL) framework that trains a single diffusion model to approximate the entire Pareto front. By using continuously varying preference weights as a conditioning signal during training, the framework enables users to navigate optimal trade-offs at inference time without retraining or maintaining multiple checkpoints.
Evaluation
ParetoSlider was evaluated on three state-of-the-art flow-matching backbones:
Results show that the single preference-conditioned model matches or exceeds the performance of baselines separately trained for fixed reward trade-offs, while uniquely providing fine-grained control over competing generation objectives.
---
*Auto-collected on 2026-04-24*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177618690