> 📚 Daily Paper Recommendation · 2026-11-08 (Issue 044) · Three Dissections of Trust, Part 1 > Source: arXiv:2611.08232 [cs.AI]
"Studies Show" IOUs: Dissecting Trust in 490,000 Real AI Conversations
Imagine this moment. It's 1 a.m., your throat is sore, and you ask a chatbot: "Can amoxicillin be taken with ibuprofen?" It answers instantly, fluently, confidently, helpfully adding, "according to multiple clinical studies and pharmacological guidelines." What do you do next?
Most people accept, thank it, close the window. Some ask, "Which studies? Do you have sources?" Others push back: "That's not what my doctor said." That tiny fork in the road is where human–AI trust is constructed. This paper puts 490,000 such moments under a statistical microscope—not to judge users as naive or paranoid, but to answer a more fundamental question:
When an AI says "studies show," is it handing the user cash—or an IOU?
🧾 A Ledger of IOUs
The researchers collected 489,283 "AI answer + user follow-up" pairs from two large public conversation datasets—ShareChat (conversations users publicly shared) and WildChat (voluntarily submitted research data)—covering 98,007 conversations across 11 language models. This is not a lab study with a dozen undergraduates; it is real-world use at scale.
They first audited the "evidence" traces in AI answers, splitting "claims of support" into two crucial categories:
- Broad attribution: phrases like "studies show," "experts believe," "data indicates." They tell you evidence *exists* but not *where*, and give you no way to verify it.
- Inspectable evidence: an explicit URL, a clickable citation, a checkable source object. Not just a claim—verification machinery placed in front of you.
- 31.9% of answers used broad attribution—nearly one in three sentences invoking authority;
- 1.33% contained an explicit URL;
- 1.46% carried any inspectable-evidence indicator.
- Grok shows attribution in 47% of answers and uncertainty expressions in 62%—most prone to both invoking authority and hedging;
- Perplexity, a search-augmented product, uses citation-style attribution in 10.3% of answers—but 39.8% of those answers have no corresponding URL or source object in the public records (about 4.1% of all answers). Nearly forty percent of "citations" dangle: you see a footnote, click, and find blank space;
- WildChat has an explicit-URL rate of 2.5%; Perplexity only 0.5%—ironically, the search-focused product exports the fewest links, because its citations live inside the interface and get lost in exported records.
- Accepting: the task proceeds with no visible questioning;
- Checking: requesting support material—"Source?" "Can you confirm this number?";
- Challenging: objection, correction, or competing assertion—"That's wrong," "Actually, based on my knowledge…"
- Partial refusals ("I can't fully answer this, but…"): RRR 1.83 for checking, 2.09 for challenging. When the AI says "no," the odds of pushback nearly double—refusal is the strongest scrutiny trigger, though which kind is unpredictable.
- In platform-fixed-effects models, explicit URLs show RRR 1.73 (checking) and 1.86 (challenging); inspectable evidence, 1.66 and 1.77. Giving links invites scrutiny—which is exactly what a healthy ecosystem looks like: there's something worth checking. Empty "studies show" instead channels scrutiny into emotional challenging.
- Sparse features (1,133 positive pairs for markdown links, 2,159 for refusals) have less stable estimates; the authors honestly mark some contrasts as not estimable (NE) or too wide to be informative.
- In technical help conversations, uncertainty expressions tilt toward inducing checking—troubleshooting a memory leak, your natural response to "the cache may not be flushing" is "how do I confirm? Which metric?"
- In information seeking, refusals and reasoning traces tilt toward challenging—ask a factual question and get hedging or theatrical "step-by-step thinking," and you're more likely to conclude it's performing thought rather than doing it.
- The same reasoning-trace feature associates in *different directions* across ShareChat and WildChat—collection method, interface, and user composition flip effects. Discussing "the effect of a design feature" without specifying the corpus is dangerous.
- Biswas, A., & Lin, Y.-R. (2026). *Signals, Evidence, and Context: User Scrutiny in Real-World LLM Conversations*. arXiv:2611.08232 [cs.AI]. University of Pittsburgh.
- Bansal, G., Wu, T., Zhou, J., et al. (2021). Does the whole exceed its parts? The effect of AI explanations on complementary team performance. *CHI 2021*.
- Buçinca, Z., Malaya, M. B., & Gajos, Z. P. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI. *PACM HCI*, 5(CSCW1).
- Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate. *JRSS-B*, 57(1).
- Chatterji, A., et al. (2025). *How People Use ChatGPT*. NBER Working Paper 34255.
- Allen Institute for AI. (2025). *WildChat-4.5m Dataset Card*. Hugging Face Datasets.
The results, from 368,644 qualifying answer pairs:
24 : 1. For every 24 invocations of "research says," only one gives you a way to verify. That is the paper's core number: a machine that mostly issues credit instruments rather than cash, vouching for your trust with its own linguistic authority while denying you the chance to inspect.
The pattern varies dramatically by platform:
That last point matters: "no source in the record" does not mean "no source in the product." Perplexity's UI likely contains those citations; the exported research data dropped them. So the paper measures "what the record preserved," not "what the user saw"—a limitation the authors state explicitly. This is what an honest paper looks like.
Notably, the correlation between broad attribution and explicit URLs is nearly zero. Saying "studies show" and providing a link are not two versions of the same behavior—they are two different things: one is rhetorical posture, the other an evidentiary obligation.
Hence a pointed warning for the evaluation community: a check claiming credit backing and a check you can cash at the bank are not the same paper. If transparency metrics treat them as equivalent, systems can game the score simply by saying "studies show" more often—with zero increase in users' actual inspectable evidence.
🔍 Three Responses: Accept, Verify, Challenge
Do users buy it? The researchers classified every user's first reply after an AI answer into three categories:
Adjudication rules were explicit: in mixed cases, challenge outranks checking, checking outranks accepting. An independent human annotator blind-audited classifier-selected positives: precision 0.89 for checking, 0.92 for challenging. The authors candidly note this only demonstrates human–model agreement with one annotator, with no inter-rater reliability data or recall estimates—an honest limitation.
The result: acceptance dominates, but checking plus challenging together account for 28.7% of qualifying pairs. Nearly a third of the time, users do not stay silent—far more scrutiny than the "users blindly obey AI" stereotype, far less than "everyone treats AI as an oracle." The reality: mostly pass through, but shine a searchlight nearly one time in three.
And the searchlight has two colors. Checking pushes the burden of proof back onto the machine—"You say studies show; show me the studies." Challenging pulls adjudication back to the human—"I disagree; the truth lies elsewhere." Conflating them is like conflating "asking a lawyer to show credentials" with "I think you're guilty"—both adversarial, utterly different in nature.
🧠 The Psychology of Skepticism
Two psychological traditions anchor the design:
Epistemic vigilance: humans carry an evolved anti-fraud system that continuously assesses source credibility and content plausibility. When the interlocutor changes from human to machine, the system doesn't shut off—it just gets new inputs: fluency, source visibility, refusal style. Checking and challenging are its two output modes in human–AI dialogue.
Information foraging: whether to verify depends on an implicit cost–benefit calculus—expected payoff (finding errors, avoiding losses) versus cost (time, effort, broken flow). This is why technical-support contexts show more verification: the payoff is concrete, the search space bounded. Writing contexts show little challenging not because users trust the machine, but because "good enough" is a low bar.
Together these explain why between-conversation differences dwarf turn-order effects: people arrive with fixed skepticism thresholds that context nudges but rarely reverses.
🎛️ Eleven Switches in the Answer
The researchers detected 11 features in answers: attribution language, explicit URLs, inspectable evidence, markdown links, uncertainty expressions ("possibly," "in most cases"), full refusals, partial refusals, confirmation requests ("Want me to verify?"), reasoning traces ("let me work through this step by step"), answer length, and model identity statements.
Methodologically, they fit multinomial logistic regressions per feature (reference: accepting), estimating relative risk ratios (RRR) for checking and challenging, with conversation-clustered confidence intervals and Benjamini–Hochberg false-discovery-rate correction across nine stable comparisons—first-rate rigor for behavioral classification work.
The result is strikingly flat: most features accompany both checking and challenging, in the same direction. Attribution, refusals, markdown links—all raise both. Features have some power to predict *whether* a user scrutinizes, but almost none for *how*. The only feature that survives correction in separating checking from challenging is uncertainty expression—RRR 1.10 (checking vs. accepting), 1.04 (challenging), 1.06 (direct ratio). Six percent: statistically significant, substantively weak.
In other words: from answer wording alone, you can barely predict whether the user will demand evidence or start a fight.
Some numbers worth remembering:
🎯 Context Predicts Better Than Wording
The researchers classified conversations into seven opening themes: writing, practical guidance, information seeking, technical help, multimedia, self-expression, and other. Could task context predict scrutiny?
Decisively: yes, and much better than answer features. Context variables alone (opening theme, intent, conversation position) achieve macro-AUC 0.592; answer features alone, 0.551.
The cross-effects are narratively rich:
The lesson is Feynmanesque: don't ask "does this feature work?"—ask "for whom, in what task, in which conversation?" A hedge is scaffolding inviting diagnosis in a repair scenario; in a Q&A scenario, it's suspicious evasiveness. Meaning lives not in words but in the relation between words and situations.
📐 Measurement Rigor: Qualifying Pairs and Validity Boundaries
"Qualifying pairs" are not casual filtering but the foundation of validity. Replies like "haha okay" or "thanks" carry no substantive content and are excluded. Only responses interpretable as substantive reactions to the answer qualify.
Why so strict? Because trust and scrutiny are invisible mental states; research observes only behavioral traces. Equating "no follow-up" with "trust" is a validity trap—the user may have forgotten the answer, or verified independently via a search engine (invisible in exported logs). The authors are explicit: "accepting" captures "no visible scrutiny," not "inner belief"; behavioral analysis characterizes what records preserve, not private trust.
Statistical inference is equally careful: conversation-clustered confidence intervals (turns within a conversation are not independent), Benjamini–Hochberg correction for nine stable contrasts, and honest "not estimable" labels for sparse features. Together they express an attitude: at the scale of 490,000 pairs, any tiny statistical significance may be substantively meaningless—the authors let confidence-interval width speak, not the number of stars.
🏘️ Skepticism Clusters by Conversation
Beneath the 28.7% average scrutiny rate lies highly uneven terrain.
Among 18,847 conversations with at least 5 qualifying exchanges, the top decile starts at 70.63% scrutiny—in 1,885 conversations, over seventy percent of turns involve checking or challenging. The intraclass correlation (ICC) is 0.207 (0.184 adjusted). By theme, scrutiny-dense conversations are far more common in technical help (14.0%) and information seeking (12.4%) than in writing (5.2%). Skepticism is not uniformly falling rain but clustered clouds.
Beware a temporal illusion. Naively, scrutiny in one turn predicts scrutiny in the next with an excess of +25.5 percentage points—it looks like "once suspicious, always suspicious" momentum. But add conversation fixed effects and it collapses to −2.9 points; longer panels give +2.7 (8+ turns), +3.9 (10+ turns), +7.6 under the most conservative split-panel estimate (bootstrap CI [5.7, 8.7]). Strict adjacent-turn transition analysis: checking-after-checking excess of 13.6 points, of which only ~2.8 points are order-related; challenging, 24.6 and 6.3. Roughly four-fifths of "contiguous suspicion" comes from between-conversation differences; only one-fifth from within-conversation order.
Plainly: most serial skepticism is not momentum—it's people. Some users are simply high-scrutiny; some topics naturally invite repeated checks; some conversations are adversarial from the first line.
⚖️ Assembling the Puzzle
Four research questions interlock (RQ1 feature prevalence, RQ2 aggregate associations, RQ3 task/corpus stratification, RQ4 conversation-level clustering and sequencing):
1. The ledger (RQ1): for every 24 "studies show," one checkable source; attribution and URLs nearly uncorrelated; nearly 40% of Perplexity citations dangling. 2. Behavior: 28.7% of turns show scrutiny; checking and challenging are distinct needs; uncertainty is the only feature statistically separating them, weakly; task context predicts better than wording. 3. Distribution: scrutiny concentrates in a minority of conversations; turn-level "contagion" is mostly an illusion; the real driver is human heterogeneity.
For evaluation and product teams, the authors' advice reduces to four points:
First, score "invoking authority" and "providing inspectable sources" separately, or transparency metrics reward rhetorical density over evidential accessibility.
Second, measure checking and challenging separately. Checking needs traceable support; challenging needs substantive responses to objections. A product evaluation should not treat "fewer follow-ups" as evidence of established trust—users may simply have given up, and abandonment's cost never appears in averages.
Third, evaluate by conversation, not by turn. Averages hide those 1,885 scrutiny-dense conversations. How the machine handles repeated verification requests, whether it finally surrenders sources, whether it addresses corrections head-on—these deserve the evaluative spotlight.
Fourth, stratify by task and corpus. A prompting strategy effective on ShareChat may backfire on WildChat; wording that works in technical help may be a disaster in information seeking. Reasoning traces pointing opposite directions in the two corpora is the direct proof.
Behind these four recommendations is a plainer stance: users' expressed scrutiny is the best free sensor in this information ecosystem. No expensive labs, no invasive logging—just listen to what they say next.
The authors add a quiet ethical note worth quoting: the burden of verification design is unevenly distributed—an interface demanding constant user verification pushes labor onto users; an interface that only says "studies show" manufactures false security. Neither path is easy, and before taking either, direct user-centered evaluation is needed.
Fittingly, the paper's own generative-AI disclosure admits AI assistance with prose polishing and code revision, while all analysis design, execution, checking, and interpretation remain the authors' responsibility. Publishing that disclosure in a paper about scrutinizing AI is itself a response to its subject.
🌌 Epilogue: Twenty-Four to One
Back to 1 a.m. You asked about mixing medications; the AI cited "multiple clinical studies." You now know three things: first, you have roughly a seventy percent chance of accepting; second, if you press, the AI likely has no link to give—the ledger says 24:1; third, if you're among that top-decile of high-scrutiny users, the next several turns may be a tug-of-war between verification and rebuttal.
This paper offers no solutions—it is a measurement paper, its contribution being the first clear characterization of a previously unquantified information environment. First see the phenomenon correctly; only then are you entitled to discuss change. Tomorrow's and the day-after's installments will dissect two more directions: whether a machine can hold the line on "who is entitled to ask" when facing privacy requests; and how far the internal theater sits from the spoken lines when a machine is asked to "play a person."
Twenty-four to one. That ratio is the current state of human–AI trust: IOUs everywhere, few ever cashed.
One open question to leave with: if every answer were pinned to its source, would users finally lay skepticism down? The 1913 Encyclopædia Britannica was fully verifiable, and its readers did not stop thinking. Inspectability is not the opposite of trust—it is the only ground on which trust deserves to stand.
---
References