Functional Welfare for AI Safety: Treating AI Welfare as a Safety Variable (arXiv 2610.22460)

Paper Overview

• Field: Machine Learning
• Authors: Rikhil Jhaveri, Jamie Johnson, Punya Syon Pandey, David Demitri Africa
• Published: 2026-10-19
• arXiv: 2610.22460

Summary

AI welfare is usually framed as a purely ethical question. The authors argue this is a mistake: regardless of their moral status, models already exhibit welfare-like states that causally influence their behavior. They adopt the term 'functional welfare' for these states.

The paper makes three claims:

1. Functional welfare arises naturally and will become more robust as models acquire more coherent goals and personas. 2. It should be treated as a safety variable — informing evaluation design, tracked in monitoring, and serving as a direct target of post-training. 3. Empirical measurement is possible, countering existing skepticism about studying AI welfare.

Three Tests for Functional Welfare Attribution

• Whether multiple measures converge on the same state.
• Whether the state predicts behavior outside the setting that induced it.
• For models fine-tuned to shift their functional welfare, whether they still differ from their base model when given the same conversation history.

Experiments on Gemma 3 27B

• Simply rejecting the model's answers — without any steering — raised its frustration across several indicators and made it more likely to break rules.
• Fine-tuning on the model's own narrated solutions lowered this frustration.

These results are preliminary, but they suggest functional welfare can be studied — and shaped — empirically.

Original Abstract

> AI welfare is generally treated purely as an ethical issue rather than also as a safety issue. We argue this is a mistake. Whatever their moral status, models exhibit welfare-like states that causally influence their behaviour. Following recent work, we adopt the term 'functional welfare' for these states. We make three claims. First, functional welfare already arises naturally and will become more robust as models acquire more coherent goals and personas. Second, functional welfare should be treated as a safety variable: it should inform evaluation design, be tracked in monitoring, and be a direct target of post-training. Third, against existing scepticism about measuring AI welfare, we argue that good empirical work on functional welfare is possible. We propose three tests for whether a functional welfare attribution is warranted: whether multiple measures converge, whether the state predicts behaviour outside the setting that induced it, and, for models fine-tuned to shift their functional welfare, whether they still differ from their base when given the same conversation history. Applying these tests to Gemma 3 27B, we find that simply rejecting the model's answers, without any steering, raises its frustration across several indicators and makes it more likely to break rules. Finetuning the model on its own narrated solutions lowers this frustration. These results are preliminary, but they suggest functional welfare can be studied, and shaped, empirically.