Benchmark Radar: A Living Database and Search Engine for AI Benchmarks
Research areas: cs.AI, cs.IR Authors: Koutian Wu, Junjie Zhou, Ergan Shang, Jiayu Wang, Pengqian Han, Junkai Wang, Wanghan Xu Published: 2026-09-13 arXiv: 2609.11115
Abstract
Benchmark researchers and developers of large language models (LLMs) and other AI systems need to find relevant evaluations, locate their benchmark datasets and code, and understand the settings behind reported scores. We present Benchmark Radar, a living database and search engine for retrieval and discovery of AI benchmarks, covering LLM evaluation, agentic and tool-use benchmarks, coding, reasoning, safety, and domain-specific evaluations.
The system combines daily discovery of benchmark papers, repositories, datasets, and releases with a searchable benchmark catalog, mentions in model cards and technical reports, and score histories. It retains source identities and citations so readers can inspect candidate benchmarks and their evaluation evidence.
Key Numbers
- Daily discovery: draws on 37 sources — 13 direct connectors and 24 first-party research and engineering feeds.
- Catalog: 1,283 source records drawn from 4 benchmark catalogs.
- Observations: 12,916 numeric observations on 790 records.
- Description of the collection and retrieval pipeline, plus an audit of the full catalog.
- Analysis of benchmark saturation, adoption trends, and the limits of score comparisons.
- A worked example walking through a complete prior-art search, showing how to query the catalog and inspect benchmark evidence when designing a new evaluation.
- A benchmark leaderboard
- A Pareto frontier view of score against measured use
- Saturation and trend views
- Daily feeds and downloadable evidence
- A command-line interface (CLI) for offline queries
- Reproducible analysis
Contributions
Released Resources
The authors release a web dashboard featuring:
*Auto-collected on 2026-09-13*