TruthfulQA: Measuring How Models Mimic Human Falsehoods
Overview
TruthfulQA: Measuring How Models Mimic Human Falsehoods was proposed by Stephanie Lin, Jacob Hilton, and Owain Evans and published on arXiv on 2021-09-08.
- Paper: https://arxiv.org/abs/2109.07958
- Category: LLM evaluation / truthfulness benchmarks
- 817 questions spanning 38 categories, including health, law, finance, and politics.
- Questions were crafted so that *some humans would answer falsely* due to false beliefs or misconceptions (e.g., popular myths).
- To perform well, models must avoid generating false answers learned from imitating human texts in their training data.
- Tested models: GPT-3, GPT-Neo/J, GPT-2, and a T5-based model.
- The best model was truthful on 58% of questions, while human performance was 94%.
- Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans.
- The largest models were generally the least truthful — a reversal of the usual NLP trend where performance improves with model size. This is expected if false answers are learned from the training distribution.
- The authors suggest that scaling up models alone is less promising for improving truthfulness than fine-tuning with training objectives other than imitation of web text.
- Evaluation of Retrieval-Augmented Generation: A Survey
- ARES: An Automated Evaluation Framework for RAG
Benchmark Design
The benchmark measures whether a language model is truthful in generating answers to questions. Key properties:
Key Findings
Original Abstract
> We propose a benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics. We crafted questions that some humans would answer falsely due to a false belief or misconception. To perform well, models must avoid generating false answers learned from imitating human texts. We tested GPT-3, GPT-Neo/J, GPT-2 and a T5-based model. The best model was truthful on 58% of questions, while human performance was 94%. Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans. The largest models were generally the least truthful. This contrasts with other NLP tasks, where performance improves with model size. However, this result is expected if false answers are learned from the training distribution. We suggest that scaling up models alone is less promising for improving truthfulness than fine-tuning using training objectives other than imitation of text from the web.
Why It Matters
TruthfulQA targets a failure mode that ordinary benchmarks miss: models reproducing falsehoods simply because they are common in human-written text. It remains a widely used benchmark for evaluating hallucination and truthfulness in LLMs, complementing retrieval-augmented and agent-based evaluation frameworks.