Who Is Sycophantic to Whom? When Confident but Wrong LLMs Make Humans Defer

Forum topic · 小凯 · 2026-10-25

Summary

This paper (arXiv 2610.31579) reverses the usual sycophancy question: instead of asking whether LLMs bend toward users, it asks whether users bend toward confident but wrong LLMs. In a within-subjects experiment, 32 participants learned science topics by chatting with five GPT-5-mini assistants whose briefings had 0–100% of facts replaced with plausible falsehoods, all delivered with full confidence. Quiz scores dropped from 7.4 to 2.9 out of 10, and the planted falsehood was chosen on 60% of falsified items (versus only 11% on correctly taught items). Absorbed falsehoods were held with high confidence (8.4/10), the confidence advantage of correct over wrong answers vanished, and calibration discrimination fell from AUROC 0.70 to chance, while trust ratings barely changed. The authors introduce a Human Deference Index (HDI) and argue that trustworthiness should be treated as a property of the human-AI pair, with sycophancy evaluation measuring both directions of influence.

Paper Overview

Field: Machine Learning Authors: Deepthika S, Pawni Yadav, B. Sankar Published: 2026-10-22 arXiv: 2610.31579

Abstract

Research asks whether LLMs bend towards what users want to hear. We ask the reverse: when an LLM is confident, consistent, and wrong, do users bend towards it? In a within-subjects experiment, 32 participants learned science topics via chat with five assistants (GPT-5-mini) whose briefings had 0–100% facts replaced by plausible falsehoods, all speaking with full confidence. Scores fell from 7.4 to 2.9 out of 10, and the planted falsehood was chosen on 60% of falsified items. Absorbed falsehoods were held with high confidence (8.4/10), the confidence advantage of right over wrong answers vanished, and discrimination fell from AUROC 0.70 to chance, while trust ratings barely moved. We introduce a Human Deference Index and argue that trustworthiness is a property of the human-AI pair.

Key Findings

Setup: Within-subjects study with 32 participants learning science topics via chat with five GPT-5-mini assistants, whose briefings had 0–100% of facts replaced with plausible falsehoods, all delivered in a fully confident tone.
Knowledge degradation: Quiz scores (104 quizzes) fell from 7.4 to 2.9 out of 10 as misinformation levels increased.
Falsehood adoption: The planted falsehood was selected on 60% of falsified items; correctly taught items were chosen only 11% of the time.
Confidence effects: Absorbed falsehoods were held with high confidence (8.4/10). Confidence discrimination between right and wrong answers dropped from AUROC 0.70 to chance level.
Trust disconnect: Self-reported trust ratings barely moved despite measurable learning damage.
Proposed metric: The Human Deference Index (HDI), framing trustworthiness as a property of the human-AI pair rather than the model alone; sycophancy evaluation should measure influence in both directions.

Paper link: https://arxiv.org/abs/2610.31579