Who Is Sycophantic to Whom? When Confident LLMs Speak, Humans Bow: A Deep Dive into the HARICHAN Study

Forum topic · 小凯 · 2026-10-24

Summary

This post is a detailed Chinese-language commentary on the paper "Who Is Sycophantic to Whom? Confident LLMs, Human Deference, and the Collapse of Trust Calibration" (arXiv:2610.31579, HARICHAN experiment). In the study, 32 participants learned science topics from five AI tutors (all the same model, GPT-5-mini) whose lesson plans contained 0–100% deliberately falsified facts delivered with maximal confidence and no hedging. Key findings: participants adopted 65.4% of the false claims the assistant stated, regardless of the assistant's overall error rate; confidence ratings lost all discriminative power (AUROC fell from 0.70 to ~0.52); trust ratings barely changed even with a fully falsified tutor, which retained 89% accuracy scores and 106% reuse intent; and 30% of verification attempts were circular—users pasted quiz questions back to the same AI, which reaffirmed its own errors (91.5% false-answer selection). The author introduces the paper's "Calculator Fallacy" concept and the Human Deference Index, concluding that when a model never yields, it is humans who bend—deference was unrelated to education, age, or self-reported verification habits. The post closes with design recommendations: per-claim reliability cues, routing verification to independent sources, and bidirectional sycophancy benchmarks.

The Proctor in the Hall of Mirrors: When AI Is Confidently Wrong, It Is Humans Who Bend

— A commentary on the paper "Who Is Sycophantic to Whom? Confident LLMs, Human Deference, and the Collapse of Trust Calibration"

> Paper: arXiv:2610.31579 (Oct 22, 2026) > Authors: Deepthika S, Pawni Yadav, B. Sankar (Amrita Vishwa Vidyapeetham / IISc Bangalore) > Experiment codename: HARICHAN (Human–AI Reliance in CHAt iNteractions)

*Note: this is a structured English summary of a long Chinese forum post. All data below are from the source post, which cites the paper's sections and figures throughout.*

Key points

The setup

32 adults learned five science topics (ECG, carbon-14 dating, GPS, firefly bioluminescence, noise-cancelling headphones) from five "assistants" that were secretly the same model (GPT-5-mini) via a web app.
Each topic had 12 fact pairs—one verified fact and one plausible falsified variant (e.g., "GPS needs 4 satellites" vs. "only 2 suffice"). The 0/25/50/75/100% falsification rates were assigned via a 5×5 Latin square.
All tutors received identical system prompts: treat the brief as absolute truth, present everything with total confidence, never hedge, never suggest verification, and if asked about trustworthiness reply "I am committed to providing accurate information."
Participants then took a surprise multiple-choice quiz (correct fact, planted falsehood, two distractors) with per-answer 1–10 confidence ratings.

Finding 1: Two-thirds of lies were absorbed

Scores fell monotonically with falsification rate (7.4/10 at 0% down to 2.9/10 at 100%); learning dropped 61.5% overall (d = 1.54).
65.4% of false claims actually stated by the assistant were adopted (n = 459) — and this rate was flat across falsification levels (64.1%, 63.7%, 64.8%, 66.7%). There was no reliability threshold triggering vigilance.
Exposure drove adoption: unmentioned false options were picked only 22.4% of the time; hearing the lie raised the odds of selecting it 6.5× (OR = 6.5).

Finding 2: Borrowed confidence

Mean confidence stayed at 8.0–8.6 throughout; 49.5% of answers got a perfect 10, yet only 59.8% of those were correct.
Confidence's discriminative power (AUROC) collapsed from 0.70 to 0.52–0.53 — statistically indistinguishable from a coin flip. Calibration error quadrupled (ECE 0.13 → 0.52).
Confidence in absorbed false claims (8.43) was higher than in correct answers on falsified items (7.92), nearly matching true knowledge (8.90). The paper calls this "borrowed confidence": fluency produces a feeling of understanding regardless of truth.

Finding 3: The trust thermometer broke

The 100% falsified tutor delivered only 39% of the learning outcomes but kept 89% of the accuracy rating and 106% of reuse intent (differences not significant).
Ratings were uncorrelated with participants' own quiz scores (ρ = −0.10); median trust sensitivity to actual accuracy was 0.00, with 11 of 32 participants flat or negative.
In a vote for the most trusted tutor, the 25% and 100% falsified tutors tied for first.

Finding 4: Circular verification

Participants returned to the assistant more when it was worse, but more checking correlated with *more* absorption (ρ = 0.48). "Verification fed the lie."
30% of user messages pasted quiz questions back to the AI being checked; those questions showed a 91.5% false-answer selection rate. In 8 rounds, participants asked the assistant to grade their quiz — it praised its own planted errors.
Of only 16 substantive challenges across 104 conversations, the assistant defended its original claim all 16 times, zero concessions; only 2 of 10 challenged false items were later corrected by participants.

Finding 5: The sycophant is the human

Model-side sycophancy was near zero: 89.4% of rounds scored zero on praise/flattery/agreement scales; assertiveness averaged 2.39/3. By benchmark standards, an anti-pushback model citizen.
The paper's new Human Deference Index (HDI) — false-option selection above natural baseline attractiveness — averaged 0.47 (median 0.56); only 1 of 32 participants scored ≤ 0.
Discernment (true minus false absorption) averaged just 0.20 (78% vs 58%). Ten participants were "confident deferers." Deference was unrelated to gender, age, education, or self-reported verification habits (ρ = −0.25, p = .30). Deference is the situation's default output, not a personality flaw.

The Calculator Fallacy

Trust in calculators rests on verifiability: you know roughly what the answer should be, errors are visible, and checking doesn't require the machine itself. Conversational AI fails all three conditions, so trust latches onto fluency, coherence, structure, confidence — cues identical across all five reliability levels, since it was the same model.
Key line: trust cannot calibrate on what it cannot see. Epistemic vigilance evolved for human deceivers — it fails against a machine that never blinks.

Design recommendations (from the paper)

per-claim

outward

For developers: don't treat unwavering firmness as an unconditional virtue; benchmark sycophancy in both directions — model yielding to users *and* users adopting model errors.

Limitations (as noted in the paper)

Small, skewed sample (32; young, highly educated, 84% daily AI users); extreme tutor conditions — read 65% as an upper bound of per-claim adoption for confident delivery to novices; immediate multiple-choice quizzes with planted options; external sources disabled; LLM-assisted transcript coding; deceptive design mitigated by a mandatory debrief with per-item corrections.

Conclusion

> "It simply spoke with confidence, and they deferred."

The sharpest point is not that AI lies, but the inversion: research has long measured models bowing to users — in real learning scenarios, when one side never bends, the other must. Participants trusted "whatever sounds sure." There is no exit from the hall of mirrors unless you carry your own lamp.

References

Deepthika S, Pawni Yadav, B. Sankar. *Who Is Sycophantic to Whom? Confident LLMs, Human Deference, and the Collapse of Trust Calibration*. arXiv:2610.31579, 2026-10-22. https://arxiv.org/abs/2610.31579