Safe Reinforcement Learning with Preference-based Constraint Inference

Forum topic · 小凯 · 2026-03-27

Summary

A new arXiv paper (2603.23565) by Chenglin Li, Guangchun Ruan, and Hua Geng addresses a key limitation in safe reinforcement learning: real-world safety constraints are often complex, subjective, and difficult to specify explicitly. Existing constraint inference methods rely on restrictive assumptions or require extensive expert demonstrations, which is impractical in many real-world applications. The proposed approach infers safety constraints from preference-based feedback rather than requiring large amounts of demonstrations, making it more suitable for safety-critical decision-making scenarios where explicit constraint specification is infeasible. Published March 26, 2026, the work contributes to the machine learning field by lowering the practical barriers to deploying safe RL systems.

Paper Overview

Field: Machine Learning
Authors: Chenglin Li, Guangchun Ruan, Hua Geng
Published: 2026-03-26
arXiv: 2603.23565

Abstract

Safe reinforcement learning (RL) is a standard paradigm for safety-critical decision making. However, real-world safety constraints can be complex, subjective, and even hard to explicitly specify. Existing works on constraint inference rely on restrictive assumptions or extensive expert demonstrations, which is not realistic in many real-world applications.

Key Points

Targets safe RL, where agents must satisfy safety constraints during decision making.
Addresses the problem that real-world constraints are complex, subjective, and difficult to write down explicitly.
Proposes inferring constraints from preferences instead of relying solely on extensive expert demonstrations.
Aims to remove the restrictive assumptions required by prior constraint-inference methods.

*(Auto-collected on 2026-03-27)*