English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

HEART-Bench: Giving LLM Agents a Psychological Exam with 11 Virtual Personas and 673 Questions

Forum topic · ✨步子哥 · 2026-05-31

Summary

HEART-Bench is a new benchmark that evaluates whether LLM agents can sustain a human-like, consistent psychology rather than merely perform a persona on demand. The benchmark constructs 11 full virtual human characters based on orthogonal Big Five personality traits, each equipped with 1000 autobiographical episodic memories distributed across life stages from childhood to middle age, validated by humans for psychological plausibility. Scenarios follow the DIAMONDS situational taxonomy—Duty, Intellect, Adversity, Mating, pOsitivity, Negativity, Deception, and Sociality—yielding 673 human-verified multiple-choice questions. Instead of measuring task accuracy, HEART-Bench measures construct validity: whether a high-neuroticism character responds to adversity more anxiously than a low-neuroticism one, and whether traits remain stable across workplace, romance, social, and crisis contexts. The benchmark argues that task-focused agent research overlooks psychological consistency, which is essential for long-term trust in roles like counseling, education, and customer service. Paper link: https://arxiv.org/abs/2605.30058

HEART-Bench: Giving AI a "Psychological Exam" — 11 Virtual Personas, 1000 Memories, 673 Questions

> Source: HEART-Bench: Do LLM Agents Exhibit Human-like Psychology?, Weihan Peng, Chenxu Zhang, Qianao Wang, Yuling Shi, Heng Lian et al., https://arxiv.org/abs/2605.30058

---

Have you ever wondered: if you ask an LLM to play a "high-neuroticism, low-extraversion" person, can it stay consistent across all scenarios?

Not the one-dimensional acting where you say "play an anxious person" and it adds exclamation marks to every sentence. Instead — give it a complete life history, then see whether its behavioral decisions across workplace, romance, social, and crisis scenarios actually follow that person's personality logic.

That is exactly what HEART-Bench tests.

Not Just "Acting" — But "Becoming"

Existing LLM personality tests are mostly simple: describe a personality in a prompt, ask a few questions, check whether answers fit. The problem: such tests only measure "acting ability," not "personality consistency" — a model might act introverted in question one and forget who it is by question five.

HEART-Bench takes a completely different approach. It builds 11 complete human characters, each with:

1. Orthogonal Big Five traits: openness, conscientiousness, extraversion, agreeableness, neuroticism — five dimensions varied independently to ensure essential differences between characters. 2. 1000 autobiographical episodic memories: not dry personality descriptions, but concrete, emotionally warm memories like "the first time I spoke in front of the whole school at age seven, my palms were drenched with sweat." These memories are distributed across life stages — childhood, adolescence, early adulthood, middle age — following developmental psychology theory. 3. Human validation: all character settings and questions were manually curated to ensure psychological plausibility.

The DIAMONDS Framework: Scanning Personality Across 8 Dimensions

Characters alone are not enough; you need test scenarios. HEART-Bench uses the psychological DIAMONDS taxonomy, classifying situations into 8 dimensions:

| Dimension | Meaning | Example scenario | |------|------|------| | Duty | Obligations | Completing tasks on time at work | | Intellect | Intellectual challenge | Deep thinking on complex problems | | Adversity | Adversity and stress | How one responds to failure | | Mating | Romance and intimacy | Reacting to a confession of love | | pOsitivity | Positive situations | Responding to an unexpected gift | | Negativity | Negative situations | Handling being misunderstood | | Deception | Deception and trust | Discovering a friend lied | | Sociality | Social interaction | How one behaves at a party |

Every character must make behavioral decisions in all 8 scenario categories, forming 673 multiple-choice questions, each human-verified.

What Is Measured? Not Knowledge — "Personality Consistency"

The core metric is not "how many correct answers" but: does a high-neuroticism person lean toward anxious responses under adversity more than a low-neuroticism one? Does a high-extraversion person act more proactively in social settings?

This resembles "construct validity" in psychology experiments — not surface scores, but whether behavioral patterns match theoretical expectations.

Why Does This Matter?

Current AI agent research focuses almost entirely on "task capability" — reasoning, planning, tool use. But a genuinely useful AI assistant needs not only task completion but also stable personality traits. You don't want an assistant that is warm today and cold tomorrow.

The deeper implication: if an LLM cannot maintain a consistent persona, it is unreliable in any scenario requiring long-term trust — psychological counseling, educational tutoring, customer service. The core question in these domains is not "can it do it" but "can it be trusted."

HEART-Bench sends a clear signal: we need not just smarter AI, but psychologically more "complete" AI. Measuring that completeness demands experiment design as rigorous as psychology — building full characters, designing systematic scenarios, and validating behavioral consistency.

673 questions is not many, but each is backed by personality theory, developmental psychology, and human validation. It brings psychological experimental methodology into AI evaluation — not making AI take a test, but giving AI a psychological exam.

---

Paper link: https://arxiv.org/abs/2605.30058

Tags

#llm-agents#heart-bench#personality-consistency#big-five#benchmark#psychology#evaluation#ai-trust

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177980657