It's High Time: A Survey of Temporal Question Answering (arXiv 2505.20243)

Overview

This post introduces a survey paper on Temporal Question Answering (Temporal QA):

• Title: It's High Time: A Survey of Temporal Question Answering
• Authors: Bhawna Piryani, Abdelrahman Abdallah, Jamshid Mozafari, Avishek Anand, Adam Jatowt
• Source: https://arxiv.org/abs/2505.20243
• Type: Survey
• Category: Time Aware Search

Temporal QA deals with questions whose answers depend on time — e.g., event ordering, durations, frequencies, and facts that change over time. The survey aims to provide a unified, comparable framework for work scattered across NLP, information retrieval, and LLM-based systems.

Key Points from the Post's Analysis

• Unified perspective: The survey organizes dispersed Temporal QA research into a comparable taxonomy, covering representation learning, retrievers, re-rankers, planners, generators, and feedback mechanisms.
• LLM integration: Temporal reasoning increasingly interfaces with RAG, tool use, and agentic search, where the model decides whether, when, and how many times to retrieve.
• Time-sensitive retrieval pipeline: The post contrasts four mainstream lines:
• *Dense retrieval* — high recall, low latency, suited to first-stage retrieval;
• *Late interaction* (e.g., ColBERT) — higher precision but larger indexes;
• *Generative IR* — documents "generated" via tokens/docids, simplifying cascades;
• *Agentic search* — search as sequential decision-making with multi-hop and self-reflection.
• Timeline: 2019–2021: BERT re-rankers and DPR established neural retrieval; 2022–2023: RAG and FreshLLM pushed retrieval-generation fusion; from 2024: conversational/agentic search and Gen-RecSys; 2025–2026: RL-trained search agents, Deep Research, and GraphRAG.

Evaluation Paradigms

Typical benchmarks and metrics discussed in this space include:

• Datasets: MS MARCO, BEIR, Natural Questions, domain corpora
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency and token cost
• Baselines: BM25, dense retrieval, cross-encoder re-ranking, retrieval-free LLMs, commercial search APIs

Takeaways for Search / RAG Practitioners

1. Architecture: Cascaded retrieve–re-rank–generate remains dominant, but agentic paradigms make retrieval strategy itself learnable. 2. Data: High-quality instruction data and click/session logs matter; synthetic data risks knowledge leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Deployment: Latency, cost, explainability, and safety are hard constraints for industrial systems.

Open Problems

The post highlights: lack of unified benchmarks, irreproducibility on private data, LLM evaluation bias, and safety/cost constraints for agentic systems. Future directions include finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, and multimodal/cross-lingual consistency.

Related Entries

• Right Answer at the Right Time — Temporal Retrieval-Augmented Generation (arXiv:2510.16715)
• Time-Sensitive Retrieval-Augmented Generation for Question Answering
• TimeR4: Time-aware Retrieval-Augmented LLMs for Temporal Reasoning

> Note: The original forum post is largely a template-based analysis built from the paper's abstract and metadata; quantitative conclusions should be verified against the original PDF.