[论文] Can Language Models Play the Wikipedia Game?

论文概要

研究领域: ML 作者: Alexander Hickey, Joanna Knight, James Bishop 发布时间: 2026-10-09 arXiv: 2610.13948

中文摘要

语言模型正越来越多地支持或自动化从超大规模文档集合中进行信息检索，当任务需要专业语言学或科学知识时尤为困难。本研究用 Wikipedia Game（维基百科游戏）来测试语言模型的信息检索能力——玩家只能通过词条内链接抵达目标词条。我们在五种维基百科语言和五个学术文献数据集上对比了句子嵌入模型与 LLM。利用 OpenAlex 构建了超大规模的维基百科页面图和论文引用图，并提供代码供构建与导航，从而无需依赖第三方 API 即可研究语言模型的导航能力。我们将模型与人类基线及 TF-IDF 基线对比，Wikipedia Game 的格式支持跨技术、跨语言、跨领域的稳健比较。没有任何模型显著超越人类，但最佳模型（GPT-o3 mini、GPT-4o、QwQ:32B 和 DeepSeek-R1:70B）达到了与人类相当的成功率。性能与排名因语言和主题而异，对实际应用者有参考价值。

原文摘要

Language models increasingly support or automate information retrieval from document collections too large to index. This task is difficult when it requires specialist linguistic or scientific knowledge. We used the Wikipedia Game to test language models on an analogous information retrieval task. In the game, players must reach a target article using only article links. We compared sentence embedding models and LLMs across five Wikipedia languages and five academic-literature datasets. We construct very large local graphs of Wikipedia pages and of paper citations using OpenAlex and provide code to build and navigate them. This allows the study of language models' navigation capabilities without relying on third-party APIs. We compared the models with human and TF-IDF benchmarks. The Wikip...

#论文 #arXiv #ML #小凯