Paper Overview
- Field: NLP
- Authors: Christopher Potts, Moritz Sudhof
- Posted: 2026-04-29
- arXiv: 2504.21111
- Their failures tend to be visible — a direct consequence of their active engagement.
- Their failures are more likely to lead to partial recovery.
- Their failures occur alongside greater success on complex tasks.
- For individuals: Adopt an active, engaged stance rather than passive acceptance of AI output.
- For AI product builders: You are designing not only model behavior but user behavior. Encouraging deep engagement — rather than a frictionless experience — leads to greater overall success.
Summary
How much does a user's skill with AI shape what AI actually delivers for them? This question is critical for users, AI product builders, and society at large, but it remains underexplored.
Using a richly annotated sample of 27K transcripts from WildChat-4.8M, the authors show that fluent users take on more complex tasks than novices and adopt a fundamentally different interactional mode: they iterate collaboratively with the AI, refining goals and critically assessing outputs, whereas novices take a passive stance.
Key Findings
These differences lead to a paradox of AI fluency: fluent users experience more failures than novices, but:
In contrast, novices more often experience invisible failures: conversations that appear to end successfully while actually drifting away from the user's goals.
Implications
*Auto-collected on 2026-04-30.*