Introduction
Multimodal large language models (MLLMs) are entering core areas of human society—job interviews, psychological counseling, human-AI collaboration. Whether these models can genuinely "read" a person's personality, rather than judging from superficial impressions, is an urgent question. A May 2026 paper titled *Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality?*—authored by researchers from the University of Tokyo, Shengda AI Research Institute, and Dalian University of Technology—directly attacks this problem: existing evaluations only check whether scores are correct, never whether the reasoning behind them is sound.
The paper makes three contributions: it proposes the Grounded Personality Reasoning (GPR) task, builds the MM-OCEAN benchmark, and designs a four-metric failure-mode analysis framework. Evaluations cover 27 MLLMs (13 closed-source, 14 open-source), with an alarming result: more than half of the correct ratings had no behavioral evidence supporting them.
Core Diagnosis: The Split Between "Getting Lucky" and "Truly Knowing"
Personality perception spans two layers: a perception layer that captures behavioral cues, and a reasoning layer that maps cues to personality dimensions. Existing evaluations stop at the perception layer—given a video, the model outputs Big Five scores, and scoring measures only correlation with human annotations. This implicitly assumes that a correct score means correct understanding.
The paper's "Prejudice Gap" fatally challenges that assumption. Across 27 models, 51% of correct ratings could not be anchored to any retrievable behavioral cue. Models gave correct answers without being able to explain how they reached them. Even top closed-source models had roughly 15% of correct ratings fall into this category—they "guessed right" but didn't know why.
The deeper implication: scores themselves are unreliable. If a model scores highly on personality evaluation, the public may mistakenly believe it has social cognition, while its reasoning chain is riddled with holes. Deploying such models in high-risk scenarios like hiring screening or psychological assessment is dangerous.
> Tip: The Big Five model, also known as OCEAN, divides personality into Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism—psychology's most robust personality framework since the 1960s.
Technical Breakdown: The GPR Task and Three-Tier Evaluation
GPR Task Design
GPR requires a three-stage chain: Rating (ranked scoring) → Reasoning (open-ended evidence-based reasoning) → Grounding (structured cue localization). All three are required, forming a complete loop from "knowing what" to "knowing why":
- T1 (Rating): rank a person's five personality dimensions from a video.
- T2 (Reasoning): explain behavioral observations and the reasoning process in natural language.
- T3 (Grounding): precisely locate the specific behavioral cues supporting the judgment in multiple-choice questions.
- Benchmark limits: 1,104 clips cannot cover the full spectrum of personality expression; the Big Five itself is contested for non-Western populations, so cross-cultural generalization demands caution.
- Adversarial gaps: MM-OCEAN's transparent annotation pipeline is a double-edged sword—malicious actors could train models to game the benchmark. Adversarial variants with dynamically generated samples are urgently needed.
- Amplified social consequences: an MLLM in hiring that equates extraversion with leadership will systematically exclude introverted candidates; a counseling assistant reasoning from stereotypes harms rather than helps. GPR opens a new dimension for AI alignment: social-cognitive alignment is as important as factual alignment—arguably more urgent.
MM-OCEAN Dataset Construction
The dataset contains 1,104 multimodal video clips and 5,320 clue-localization MCQs. All annotations were produced by a five-stage "multi-agent collaboration + human verification" pipeline: an Observer agent extracts atomic behavioral cues, a Psychologist agent performs trait inference, an Examiner agent generates MCQs, a Calibrator agent aligns formatting, with human checks at multiple stages.
Each clip carries timestamped behavioral observations, evidence-anchored trait analysis, and MCQs spanning seven cognitive abilities: cue identification, temporal localization, trait mapping, causal reasoning, comparative analysis, contextual inference, and integrative judgment.
Failure-Mode Analysis Framework
Four sample-level metrics pinpoint exactly where the reasoning chain breaks:
| Metric | Meaning | Diagnostic Focus | |--------|---------|------------------| | Prejudice Rate (PR) | Correct rating without supporting cues | Guessing via stereotypes or pattern matching | | Confabulation Rate (CR) | Fabricating behavioral evidence in reasoning | Inventing facts to justify conclusions | | Integration-failure Rate (IR) | Broken logical chain between cues and conclusions | Finding cues but failing to derive them correctly | | Holistic-grounding Rate (HR) | Rating, reasoning, and grounding all correct | The ultimate measure of truly "knowing" |
Experimental Findings: A Brutal Data Picture
Low Holistic Grounding
Across 27 MLLMs, the highest HR was only 33.5%; the lowest was 0%. Even the best model failed to complete the full perception-to-reasoning loop on two-thirds of samples. Most models scored below 10%.
"Confident Raters" vs. "Cautious Reasoners"
Confident raters excel at the rating stage with high ranking accuracy, but collapse at reasoning and grounding. They pattern-match from surface features; asked to explain, they hedge or fabricate. High prejudice rates reveal their correct answers come from luck or stereotype.
Cautious reasoners perform modestly at rating, sometimes deliberately lowering output confidence, yet show relatively complete reasoning chains. They can identify some behavioral cues but struggle to integrate them into precise scores.
The coexistence of both archetypes exposes a structural defect: a hard gap separates perception from reasoning.
Closed-Source vs. Open-Source
Closed-source models outperform open-source models overall, especially at T1—but this advantage sharply erodes at T2 and T3. Even GPT-4o and Claude-3.5-Sonnet achieve HRs of only around 30%. Stronger general reasoners among open-source models (e.g., Qwen2.5-VL, InternVL2.5) perform relatively well, confirming that general reasoning ability underpins social cognition.
Key finding: reasoning ability correlates positively with grounding rate, but the slope is far below expectation. Being "smart" does not directly translate into "understanding people." Social cognition has unique cognitive-architecture demands beyond a linear extension of general reasoning.
Limitations and Open Questions
Conclusion
The "Prejudice Gap" cuts through the veneer of MLLM personality perception: 51% of correct ratings lack support, and holistic grounding rates span 0%–33.5%. If the technical community keeps numerical scores as the sole yardstick, it will continue deceiving itself in the critical domain of social cognition.
GPR, MM-OCEAN, and the four-metric failure analysis form an actionable improvement framework. The direction forward is clear: models must learn to "know and know why," moving from pattern matching toward evidence-anchored reasoning. The ultimate test of personality perception is not the score, but whether a model can assemble a personality from behavioral fragments—and honestly state which fragments support which judgments.
Guessing right was never truly knowing. True knowing means explaining why.
---
Reference
Kang, C., Yan, T., Gong, S., Zhang, M., Ouyang, L., Liu, R., Zheng, B., Lu, H., Zhang, K., Sato, Y., & Huang, Y. (2026). Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality? arXiv preprint arXiv:2605.22109.