When Are Likely Answers Right? On Sequence Probability and Correctness in LLMs

Forum topic · 小凯 · 2026-06-27

Summary

This forum post shares an arXiv paper (2606.27359) by Johannes Zenn and Jonas Geiping titled 'When are likely answers right? On Sequence Probability and Correctness in LLMs.' Many decoding methods for large language models work by shifting probability mass toward outputs the model considers more likely, either locally at the token level (e.g., greedy or beam-style decoding) or globally at the sequence level. Their effectiveness therefore hinges on a fundamental question: when does sequence probability—the conditional probability of a continuation given a prompt—actually align with correctness? The paper quantifies this relationship empirically, examining how well sequence probability predicts answer correctness across different decoding methods, models, and tasks. The authors find that the alignment between probability and correctness varies substantially depending on the decoding approach, the model, and the task domain. The results have implications for designing and choosing decoding strategies, since methods that rely on probability mass concentration only help when likelihood correlates with being right. The post was auto-collected on 2026-06-27.

Paper Overview

Field: ML Authors: Johannes Zenn, Jonas Geiping Published: 2026-06-27 arXiv: 2606.27359

Abstract

Many decoding methods for large language models can be understood as shifting probability mass toward outputs that are more likely under the model, either locally at the token level or globally at the sequence level. Therefore, their success depends on a fundamental question: when does sequence probability, that is, the conditional probability of a continuation given a prompt, actually align with correctness?

In this paper, the authors set out to quantify this relationship across decoding methods, models, and tasks.

Key Points

Many LLM decoding methods effectively concentrate probability mass on more likely outputs (token-level or sequence-level).
The usefulness of such methods depends on whether sequence probability correlates with answer correctness.
The paper empirically quantifies the probability–correctness relationship across multiple decoding methods, models, and tasks.
The relationship is found to vary considerably depending on decoding method, model, and task.

Significance

Understanding when likelihood aligns with correctness helps explain why certain decoding strategies improve performance in some settings but not others, informing better decoding design and model evaluation.

--- *Auto-collected on 2026-06-27*