Paper: When Are Likely Answers Right? On Sequence Probability and Correctness in LLMs

Paper Overview

• Field: Machine Learning
• Authors: Johannes Zenn, Jonas Geiping
• Date: 2026-06-27
• arXiv: 2606.27359

Abstract

Many decoding methods for large language models can be understood as shifting probability mass toward outputs that are more likely under the model, either locally at the token level or globally at the sequence level. Therefore, their success depends on a fundamental question: when does sequence probability, that is, the conditional probability of a continuation given a prompt, actually align with correctness?

In this paper, the authors set out to quantify this relationship across decoding methods, models, and tasks. Their analysis shows that the alignment between sequence probability and correctness varies widely depending on the setting — a key consideration for understanding why probability-shifting decoding methods work well in some contexts but not others.

Key Points

• Many LLM decoding methods implicitly assume that more probable continuations are more likely to be correct.
• The paper quantifies the relationship between sequence probability (conditional probability of a continuation given a prompt) and actual correctness.
• The probability–correctness relationship differs substantially across decoding methods, models, and tasks.
• These findings help explain when and why probability-based decoding strategies succeed or fail.