Does Theory of Mind Improvement Really Benefit Human-AI Interactions? A Study on LLM ToM Enhancement

Forum topic · 小凯 · 2026-05-19

Summary

This paper (arXiv:2505.10890) by Nanxu Gong, Zixin Chen, and Haotian Li examines whether improvements in Large Language Models' (LLMs) Theory of Mind (ToM) capability actually translate into better human-AI interactions. Existing benchmarks typically measure ToM via story-reading and third-person multiple-choice questions, overlooking the first-person, dynamic, and open-ended nature of real human-AI interactions. The authors propose a new paradigm of interactive ToM evaluation featuring both a perspective shift and a metric shift. They systematically evaluate four representative ToM enhancement techniques across four real-world datasets and a user study, covering goal-oriented tasks (e.g., coding, math) and experience-oriented tasks (e.g., counseling). Their key finding is that gains on static benchmarks do not always lead to better performance in dynamic human-AI interactions. The paper argues that interaction-based evaluation is necessary when developing next-generation socially aware LLMs for human-AI symbiosis.

Paper Overview

Research Area: Machine Learning Authors: Nanxu Gong, Zixin Chen, Haotian Li Published: 2025-05-15 arXiv: 2505.10890

Abstract (English)

Improving the Theory of Mind (ToM) capability of Large Language Models (LLMs) is crucial for effective social interactions between these AI models and humans. However, existing benchmarks often measure ToM capability improvement through story-reading, multiple-choice questions from a third-person perspective, while ignoring the first-person, dynamic, and open-ended nature of human-AI (HAI) interactions. To directly examine how ToM improvement techniques benefit HAI interactions, the authors first propose a new paradigm of interactive ToM evaluation with both perspective and metric shifts. Following this paradigm, they conduct a systematic study of four representative ToM enhancement techniques using four real-world datasets and a user study, covering both goal-oriented tasks (e.g., coding, math) and experience-oriented tasks (e.g., counseling).

Key Findings

Improvements on static ToM benchmarks do not always translate into better performance in dynamic human-AI interactions.
Third-person, story-based evaluation does not reflect the first-person, interactive nature of real HAI scenarios.
The study evaluates four representative ToM enhancement techniques across goal-oriented tasks (coding, math) and experience-oriented tasks (counseling).
Interaction-based evaluation is necessary when developing next-generation socially aware LLMs designed for human-AI symbiosis.

*Auto-collected on 2026-05-19.*