Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests

Source: https://arxiv.org/abs/2601.17617 · Type: Academic paper · Section: Agentic Search

One-line summary: A large-scale log analysis of how LLM-powered search agents actually behave — their intents, session trajectories, and how retrieved evidence shapes subsequent queries — based on 14.44M real search requests from DeepResearchGym.

Background & Motivation

LLM-powered search agents are increasingly used for multi-step information seeking, yet the IR community lacks empirical understanding of how agentic search sessions unfold and how retrieved evidence is reflected in later queries. This paper addresses that gap by analyzing real-world agentic search logs at scale, rather than relying on synthetic benchmarks.

Data & Method

• Data: 14.44M search requests forming 3.97M sessions, collected from DeepResearchGym, an open-source search API accessed by external agentic clients.
• Sessionization & annotation: Logs are sessionized; session-level intents and step-wise query-reformulation labels are assigned using LLM-based annotation.
• New metric — CTAR (Context-driven Term Adoption Rate): quantifies whether newly introduced query terms are lexically traceable to previously retrieved evidence.

Key Findings

1. Sessions are short and fast: Over 90% of multi-turn sessions contain at most ten steps, and 89% of inter-step intervals fall under one minute. 2. Behavior varies by intent: Fact-seeking sessions exhibit high repetition that increases over time, while sessions requiring reasoning sustain broader exploration. 3. Query reformulations track retrieved evidence: On average, 54% of newly introduced query terms appear in the accumulated evidence context, with additional traceability to earlier steps beyond the most recent retrieval.

Implications

The findings provide candidate signals for:

• Repetition-aware stopping policies
• Intent-adaptive retrieval budgeting
• Explicit cross-step context tracking in agentic pipelines

For the broader search/rec/personalization community, the work highlights that agentic paradigms make retrieval count and strategy themselves learnable design choices, and that process-level metrics (citation accuracy, multi-hop chain integrity) increasingly complement static nDCG-style evaluation.

Open Problems & Limitations

Typical concerns for this line of work include evaluation credibility, latency and cost, hallucination and safety in open-web retrieval, and cross-lingual/multimodal generalization. Future directions include more efficient test-time compute allocation and deeper integration with knowledge graphs and structured databases.

Resources

The anonymized logs are publicly released at a HuggingFace repository: https://huggingface.co/datasets/cx-cmu/deepresearchgym-agentic-search-logs

Related Entries

• AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforcement Learning (arXiv 2509.24193)
• Agentic Information Retrieval (arXiv 2410.09713)
• Synergizing RAG and Reasoning: A Systematic Review (arXiv 2504.15909)
• A Systematic Framework for Enterprise Knowledge Retrieval (arXiv 2512.05411)
• Retrieval Augmented Generation and Understanding in Vision: A Survey (arXiv 2503.18016)

> Note: The editorial summary above is based on the paper's abstract and public metadata; verify quantitative details against the PDF when citing specific numbers.