TopBench: A Benchmark for Implicit Prediction and Reasoning in Table Question Answering

Forum topic · 小凯 · 2026-05-04

Summary

TopBench is a new benchmark for evaluating large language models on implicitly predictive table question answering. While most table QA tasks can be solved via information extraction or simple aggregation, many real-world queries require inferring unobserved answers from historical patterns instead of mere retrieval. Developed by An-Yang Ji, Jun-Peng Jiang, De-Chuan Zhan, and Han-Jia Ye, TopBench contains 779 samples spanning four sub-tasks: single-point prediction, decision making, treatment effect analysis, and complex filtering, requiring models to produce outputs that span both reasoning text and structured tables. Experiments across text-based and agentic workflows show that current models often struggle with intent recognition, defaulting to simple lookups rather than prediction. The analysis indicates that accurate intent disambiguation is a prerequisite for eliciting predictive behavior, and that raising the ceiling of prediction accuracy requires integrating more sophisticated modeling or reasoning capabilities. Paper: arXiv 2604.28076.

Paper Overview

Field: Natural Language Processing Authors: An-Yang Ji, Jun-Peng Jiang, De-Chuan Zhan, Han-Jia Ye Published: 2026-04-30 arXiv: 2604.28076

Abstract

Large Language Models have advanced Table Question Answering, where most queries can be answered by extracting information or simple aggregation. However, a common class of real-world queries is implicitly predictive, requiring inference of unobserved answers from historical patterns rather than mere retrieval. These queries introduce two challenges: recognizing latent intent and reliable predictive reasoning over massive tables.

We introduce TopBench, a benchmark of 779 samples across four sub-tasks ranging from single-point prediction to decision making, treatment effect analysis, and complex filtering. The tasks require models to generate outputs spanning both reasoning text and structured tables. We evaluate diverse models under both text-based and agentic workflows.

Findings

Current models often struggle with intent recognition, defaulting to lookups instead of prediction.
Accurate intent disambiguation is the prerequisite for eliciting these predictive behaviors.
Improving the ceiling of prediction accuracy requires integrating more complex modeling or reasoning capabilities.