Summary
This arXiv paper (2509.00010) by Qing Zhang, Yifei Huang, and Juyoung Lee addresses a failure mode called the Fluency Trap: as generative AI makes polished prose cheap to produce, users trust fluent hallucinations while discounting accurate content once it is disclosed as AI-generated. The authors argue that binary 'Made with AI' labels rely on authorship disclosure without showing what supports a claim. They propose Provenance Density, an evidence-visualization interface showing the density of verified claims within a text. In a user study with 81 participants, an idealized Provenance Density interface produced a large discernment gap between truth and fabrication (+4.15 points, d=1.82), while participants with no signal showed no detectable discrimination. A technical audit of 200 samples found retrieval density alone is insufficient; a Consistency Veto mechanism unexpectedly carried most of the discriminative signal on dynamically queried content. The paper concludes that effective transparency must shift from authorship disclosure to evidence visualization as AI-generated content becomes indistinguishable from human writing.
Paper Overview
Field: AI/ML
Authors: Qing Zhang, Yifei Huang, Juyoung Lee
Published: 2026-09-06
arXiv: 2509.00010
Summary
As generative AI makes polished prose cheap to produce, users can no longer rely on fluency as a proxy for truth. The authors call this failure mode the Fluency Trap: users trust fluent hallucinations while also discounting accurate content once it is disclosed as AI-generated. Binary 'Made with AI' labels respond with authorship disclosure, but they do not show what supports a claim.
The paper proposes Provenance Density, an evidence-visualization interface that shows the density of verified claims in a text.
Key Findings
- In a user study with 81 participants, an idealized Provenance Density interface produced a large discernment gap between truth and fabrication (+4.15 points, d=1.82), whereas participants given no signal showed no detectable discrimination.
- A technical audit with 200 samples shows that retrieval density alone is insufficient; unexpectedly, the Consistency Veto carried most of the discriminative signal on dynamically queried content.
- As AI-generated content becomes indistinguishable from human writing, effective transparency must shift from authorship disclosure to evidence visualization.
Links
- arXiv: https://arxiv.org/abs/2509.00010
---
*Auto-collected on 2026-09-06*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178634534