Deep Retrieval at CheckThat! 2025: Identifying Scientific Papers from Implicit Social Media Mentions via Hybrid Retrieval and Re-Ranking

Authors: Pascal J. Sager, Ashwini Kamaraj, Benjamin F. Grewe, Thilo Stadelmann

Source: https://arxiv.org/abs/2505.23250 (arXiv, May 2025)

Overview

This work describes the Deep Retrieval team's system for the CheckThat! 2025 shared task, which focuses on identifying the scientific papers implicitly referenced in social media posts. Given a short, informal social media mention, the task is to link it to the correct scientific publication in a large corpus — a challenging problem because such mentions often omit titles, author names, or other precise metadata, and may paraphrase the paper's content.

System Description

The system follows a hybrid retrieval and re-ranking pipeline:

1. Candidate retrieval — combining sparse (lexical, e.g., BM25-style) and dense (semantic vector) retrieval to maximize candidate coverage; 2. Re-ranking — a second-stage model refines the retrieved candidates to improve final linkage precision.

This cascade design reflects a standard trade-off in large-scale search systems: the retrieval stage prioritizes recall, while the re-ranking stage prioritizes precision.

Positioning in the Hybrid Search vs. Vector Search Landscape

The paper is a practical data point in the ongoing debate between pure vector search and hybrid retrieval. It illustrates several themes common to modern ranking systems:

• Hybrid (sparse + dense) retrieval is often more robust than either approach alone, particularly for noisy, short queries like social media text;
• Re-ranking with stronger (typically cross-encoder-style or LLM-based) models on a small candidate set remains a dominant architecture;
• Engineering constraints such as latency, cost, and index maintenance shape how much of the pipeline can be deepened.

Notes on Evaluation

Specific benchmark numbers, datasets, and ablation results are not reproduced here. Readers citing quantitative conclusions should consult the original PDF at the arXiv link above.

Related Reading

• Azure AI Search: Outperforming vector search with hybrid retrieval
• COS-Mix: Cosine Similarity and Distance Fusion for Improved Information Retrieval (arXiv:2406.00638)
• Domain-specific Question Answering with Hybrid Search (arXiv:2412.03736)
• Modernizing Facebook Scoped Search: Keyword and Embedding Hybrid Retrieval (arXiv:2509.13603)
• Hybrid Hierarchical Retrieval for Open-Domain Question Answering
• Efficient Knowledge Graph Construction and Retrieval from Unstructured Data (arXiv:2507.03226)