Paper Overview
Research area: ML Authors: Yaoxi Shi, Cathy Mengying Fang, Pattie Maez Published: 2025-06-01 arXiv: 2506.00636
Summary
Public discourse and emerging policy typically assume that AI emotional support is a deliberate act: a lonely user consciously seeking comfort from a dedicated companion chatbot. This paper, drawing on emerging empirical evidence, argues that this picture is inaccurate on two accounts — both in how AI emotional support arises and in how it shapes future behavior.
First, AI emotional support commonly emerges incidentally within task-oriented interactions on general-purpose platforms, much as workplace friendships deepen through collaboration.
Second, these incidental encounters are path-dependent: positive experiences of AI emotional support update people's beliefs about AI's emotional capabilities and redirect their choices for future emotional support, increasing preference for AI and decreasing preference for humans.
The authors review recent evidence, including a large-scale longitudinal study conducted in collaboration with OpenAI, showing that talking with AI about personal matters for 5 minutes a day over 28 days led to a 10.3% decrease in preference for seeking support from people and an 11.6% increase in preference for AI.
Implications
These findings suggest that current policies focused on companion apps and isolated interactions cannot adequately protect human connection. Instead, effective regulation should extend to general-purpose AI systems and address the cumulative, trajectory-level changes in how people seek support. Recognizing how people stumble into AI emotional support — and how such encounters redirect human connection over time — is essential for safeguarding human well-being.
Original Abstract
> Public discourse and emerging policy typically assume that AI emotional support is a deliberate act: a lonely user consciously seeking comfort from a dedicated companion chatbot. In this paper, we draw on emerging empirical evidence and argue that this picture is inaccurate on two accounts, both in how AI emotional support arises and how it shapes future behavior. First, AI emotional support commonly emerges incidentally within task-oriented interactions on general-purpose platforms, much as workplace friendships deepen through collaboration. Second, these incidental encounters are path-dependent: positive experiences of AI emotional support update people's beliefs about AI's emotional capabilities and redirect their choices for future emotional support, increasing preference for AI and de...
--- *Auto-collected on 2026-06-05*