Safe Meta-Reinforcement Learning via Information Space Reachability

Forum topic · 小凯 · 2026-09-16

Summary

Researchers Zeyang Li, Sunbochen Tang, and Navid Azizan propose a safe meta-reinforcement learning (meta-RL) framework that explicitly accounts for safety during task adaptation. Meta-RL allows agents to adapt to unseen tasks with limited experience, but safety requirements have been underexplored in prior work. The key insight is to reason about safety in the information space, which jointly captures the physical state and the agent's belief over the underlying task. Within this space, the authors introduce a safety value function measuring the probability that the agent avoids unsafe regions indefinitely, and prove it satisfies a self-consistency condition and a Bellman equation, making it learnable via meta-RL. Building on this formulation, they develop an algorithm that learns the safety value function and uses it for safety filtering and constrained policy optimization. Experiments on meta-RL benchmarks demonstrate the effectiveness of the approach. Paper available at arXiv:2609.15915.

Paper Overview

Field: Machine Learning Authors: Zeyang Li, Sunbochen Tang, Navid Azizan Published: 2026-09-14 arXiv: 2609.15915

Abstract

Meta-reinforcement learning (meta-RL) enables agents to adapt to unseen tasks with limited experience. Despite its promise, the application of meta-RL in real-world tasks is hindered by safety requirements, which have been underexplored in prior work. This paper proposes a safe meta-RL framework that explicitly accounts for safety during adaptation.

Key Idea

The core insight is to reason about safety in the information space, which captures both the physical state and the agent's belief over the underlying task. Within this space, the authors introduce a safety value function that measures the probability of the agent avoiding unsafe regions indefinitely.

Contributions

Formalize safety in meta-RL via a safety value function defined over the information space
Prove that this function satisfies a self-consistency condition and a Bellman equation, making it learnable via meta-RL
Develop a safe meta-RL algorithm that learns the safety value function and uses it for safety filtering and constrained policy optimization
Validate the approach on meta-RL benchmarks

Original Abstract

> Meta-reinforcement learning (meta-RL) enables agents to adapt to unseen tasks with limited experience. Despite its promise, the application of meta-RL in real-world tasks is hindered by safety requirements, which have been underexplored in prior work. In this paper, we propose a safe meta-RL framework that explicitly accounts for safety during adaptation. Our key insight is to reason about safety in the information space, which captures both the physical state and the agent's belief over the underlying task. Within this space, we introduce a safety value function that measures the probability of the agent avoiding unsafe regions indefinitely. We show that this function satisfies a self-consistency condition and a Bellman equation, which make it learnable via meta-RL.

---

Paper: https://arxiv.org/abs/2609.15915