Long-form factuality in large language models
Meta information
| Field | Content | |------|------| | Title | Long-form factuality in large language models | | Authors / Affiliation | Jerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu, Nathan Hu, Jie Huang, et al. (12 authors) | | Published | 2024-03-27 | | Source | https://arxiv.org/abs/2403.18802 | | Type | Academic paper | | Section | FreshLLM and similar architectures (LLM and large-scale search) |
One-line summary
This work benchmarks and automatically evaluates the long-form factuality of LLMs in open domains.
Background and motivation
In large-scale search, recommendation, and personalization systems, agentic search has long faced challenges around efficiency, scalability, and user-intent understanding. Traditional pipeline approaches often treat retrieval, ranking, and generation in isolation, making it hard to meet LLM-era demands for natural-language interaction, multi-hop reasoning, and up-to-date knowledge. *Long-form factuality in large language models* was proposed in this context to systematically advance the theoretical and practical boundaries of this intersection.
From the problem definition, the paper's core scenario is: LLMs often generate content containing factual errors when responding to fact-seeking prompts on open-ended topics. The English abstract states:
> Large language models (LLMs) often generate content that contains factual errors when responding to fact-seeking prompts on open-ended topics. To benchmark a model's long-form factuality in open domains, we first use GPT-4 to generate LongFact, a prompt set comprising thousands of questions spanning 38 topics. We then propose that LLM agents can be used as automated evaluators for long-form factuality through a method which we call Search-Augmented Factuality Evaluator (SAFE)...
Core contributions
- A unified perspective on benchmarking and evaluating long-form factuality.
- SAFE (Search-Augmented Factuality Evaluator): an LLM agent that breaks a long-form response into individual facts and verifies each via multi-step reasoning with Google Search queries.
- LongFact: a GPT-4–generated prompt set of thousands of questions spanning 38 topics.
- An extended F1 score balancing precision (percentage of supported facts) against recall (percentage of provided facts relative to a user-preferred response length hyperparameter).
- Empirical validation: on a set of ~16k individual facts, SAFE agrees with crowdsourced human annotators 72% of the time; on a random subset of 100 disagreement cases, SAFE wins 76% of the time — while being more than 20x cheaper than human annotators.
- A benchmark of thirteen language models across four families (Gemini, GPT, Claude, PaLM-2), finding that larger models generally achieve better long-form factuality.
- Benchmark: LongFact (thousands of prompts, 38 topics), generated with GPT-4.
- Human agreement study: ~16k individual facts; SAFE agreed with crowdsourced annotators 72% of the time; in 100 randomly sampled disagreement cases, SAFE won 76%.
- Cost: SAFE is more than 20x cheaper than human annotators.
- Model comparison: thirteen models across Gemini, GPT, Claude, and PaLM-2 families; larger language models generally achieve better long-form factuality.
- Enhancing Noise Robustness of Retrieval-Augmented Language Models (arXiv:2405.20978)
- FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation (arXiv:2310.03214)
- Gorilla: Large Language Model Connected with Massive APIs (arXiv:2305.15334)
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Tasks (arXiv:2212.10509)
- ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning (arXiv:2503.19470)
- Search-R1: Training LLMs to Reason and Leverage Search Engines with RL (arXiv:2503.09516)
- Original paper: Long-form factuality in large language models. arXiv: https://arxiv.org/abs/2403.18802
Method / system architecture
1. Input and representation: fact-seeking prompts on open-ended topics; responses are decomposed into a set of individual atomic facts. 2. Core module (SAFE): an LLM agent runs a multi-step reasoning process — sending search queries to Google Search and determining whether each fact is supported by the search results. 3. Evaluation metric: an extended F1 score that balances precision (supported-fact percentage) and recall (provided-fact percentage relative to a desired response length).
LongFact, SAFE, and all experimental code are available at https://github.com/google-deepmind/long-form-factuality.
Experiments and evaluation
Note: for exact quantitative tables, refer to the original PDF; this post is based on the abstract and public metadata.
Key conclusions and insights
1. Architecture: cascaded retrieve-rerank-generate remains mainstream, but the agentic paradigm makes retrieval policy itself learnable and enables verifiable generation via search. 2. Data: high-quality instruction data and click/session logs remain critical; synthetic data requires guarding against leakage and distribution shift. 3. Evaluation: LLM-as-judge approaches (like SAFE) can outperform crowdsourced annotators but should be cross-validated with human review. 4. Product: latency, cost, interpretability, and safety are hard constraints for industrial deployment — optimizing academic benchmarks alone is insufficient.
Limitations and future work
Possible limitations include experiment scale bounded by compute budget, mismatch between benchmarks and real user distributions, English-centric data with unknown cross-lingual generalization, and safety risks of agent systems operating on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints in recommendation settings.
Related entries (cross-references)
Original abstract
> Large language models (LLMs) often generate content that contains factual errors when responding to fact-seeking prompts on open-ended topics. To benchmark a model's long-form factuality in open domains, we first use GPT-4 to generate LongFact, a prompt set comprising thousands of questions spanning 38 topics. We then propose that LLM agents can be used as automated evaluators for long-form factuality through a method which we call Search-Augmented Factuality Evaluator (SAFE). SAFE utilizes an LLM to break down a long-form response into a set of individual facts and to evaluate the accuracy of each fact using a multi-step reasoning process comprising sending search queries to Google Search and determining whether a fact is supported by the search results. Furthermore, we propose extending F1 score as an aggregated metric for long-form factuality. To do so, we balance the percentage of supported facts in a response (precision) with the percentage of provided facts relative to a hyperparameter representing a user's preferred response length (recall). Empirically, we demonstrate that LLM agents can outperform crowdsourced human annotators - on a set of ~16k individual facts, SAFE agrees with crowdsourced human annotators 72% of the time, and on a random subset of 100 disagreement cases, SAFE wins 76% of the time. At the same time, SAFE is more than 20 times cheaper than human annotators. We also benchmark thirteen language models on LongFact across four model families (Gemini, GPT, Claude, and PaLM-2), finding that larger language models generally achieve better long-form factuality. LongFact, SAFE, and all experimental code are available at https://github.com/google-deepmind/long-form-factuality.