Safe Meta-Reinforcement Learning via Information Space Reachability

Forum topic · 小凯 · 2026-09-16

Summary

Researchers Zeyang Li, Sunbochen Tang, and Navid Azizan propose a safe meta-reinforcement learning framework that explicitly accounts for safety during adaptation to unseen tasks. The key insight is to reason about safety in the information space, which captures both the physical state and the agent's belief over the underlying task. Within this space, the authors introduce a safety value function measuring the probability that the agent avoids unsafe regions indefinitely. They prove this function satisfies a self-consistency condition and a Bellman equation, making it learnable via meta-RL. Building on this formalism, they develop a safe meta-RL algorithm that learns the safety value function and uses it for safety filtering and constrained policy optimization. Experiments on meta-RL benchmarks demonstrate the effectiveness of the approach. Paper: arXiv 2609.15915.

Paper Overview

Research Area: Machine Learning Authors: Zeyang Li, Sunbochen Tang, Navid Azizan arXiv: 2609.15915

Abstract

Meta-reinforcement learning (meta-RL) enables agents to adapt to unseen tasks with limited experience. Despite its promise, the application of meta-RL in real-world tasks is hindered by safety requirements, which have been underexplored in prior work. In this paper, the authors propose a safe meta-RL framework that explicitly accounts for safety during adaptation.

The key insight is to reason about safety in the information space, which captures both the physical state and the agent's belief over the underlying task. Within this space, a safety value function is introduced that measures the probability of the agent avoiding unsafe regions indefinitely. The authors show that this function satisfies a self-consistency condition and a Bellman equation, which make it learnable via meta-RL.

Method

Based on this formalism, the authors develop a safe meta-RL algorithm that:

Learns the safety value function in information space
Uses it for safety filtering
Incorporates it into constrained policy optimization

Results

Experiments on meta-RL benchmarks demonstrate the effectiveness of the proposed method in maintaining safety during adaptation.