TopBench: A Benchmark for Implicit Prediction and Reasoning in Table Question Answering

Forum topic · 小凯 · 2026-05-04

Summary

TopBench is a new benchmark from researchers at Nanjing University (An-Yang Ji, Jun-Peng Jiang, De-Chuan Zhan, Han-Jia Ye) evaluating large language models on implicit predictive table question answering. While most Table QA queries can be answered via extraction or simple aggregation, many real-world queries require inferring unobserved answers from historical patterns. TopBench contains 779 samples across four sub-tasks: single-point prediction, decision making, treatment effect analysis, and complex filtering, requiring outputs that span reasoning text and structured tables. Experiments across text-based and agentic workflows show that current LLMs often fail at intent recognition, defaulting to simple lookups instead of prediction. The authors find that accurate intent disambiguation is a prerequisite for eliciting predictive behavior, and that raising the ceiling of prediction accuracy requires integrating more complex modeling or reasoning capabilities. Paper: arXiv 2604.28076.

Paper Overview

Field: Natural Language Processing Authors: An-Yang Ji, Jun-Peng Jiang, De-Chuan Zhan, Han-Jia Ye Published: 2026-04-30 arXiv: 2604.28076

Abstract

Large Language Models have advanced Table Question Answering, where most queries can be answered by extracting information or simple aggregation. However, a common class of real-world queries is implicitly predictive, requiring inference of unobserved answers from historical patterns rather than mere retrieval. These queries introduce two challenges: recognizing latent intent and reliable predictive reasoning over massive tables.

We introduce TopBench, a benchmark of 779 samples across four sub-tasks ranging from single-point prediction to decision making, treatment effect analysis, and complex filtering. Experiments reveal that current models often struggle with intent recognition, defaulting to lookups. Accurate intent disambiguation is the prerequisite for eliciting these predictive behaviors. Raising the ceiling of prediction accuracy requires integrating more complex modeling or reasoning capabilities.

Key Findings

Implicitly predictive queries are a common real-world class of table questions that require inference beyond retrieval.
TopBench covers 779 samples and four sub-tasks: single-point prediction, decision making, treatment effect analysis, and complex filtering.
Current models frequently fail at intent recognition, defaulting to simple lookups instead of performing prediction.
Intent disambiguation is identified as the prerequisite for predictive behavior; improving accuracy further requires stronger modeling or reasoning capabilities.
Evaluation was conducted under both text-based and agentic workflows.

--- *Originally posted on zhichai.net, auto-collected 2026-05-04.*