Investigating ChatGPT Search: Insights from 80 Million Clickstream Records (Semrush, Feb 2025)
This post indexes a Semrush blog study analyzing 80 million clickstream records to understand how people use ChatGPT Search, published in February 2025.
- Original source: https://www.semrush.com/blog/chatgpt-search-insights/
- The study leverages large-scale clickstream data (80 million records) to empirically examine user behavior on ChatGPT's web search feature.
- It situates ChatGPT Search within the broader evolution of information retrieval: from BM25 and dense retrieval through cross-encoder reranking to LLM-driven, agentic search.
- The accompanying forum analysis highlights how LLM-era systems reassign responsibilities across retrieval, reranking, generation, and tool calling, making the number and strategy of retrieval steps themselves learnable.
- Evaluation is shifting from static ranking metrics (nDCG, MRR, Recall@k) toward task success rate, citation accuracy, multi-hop reasoning completeness, latency, and token cost.
Key points
Engineering takeaways
| Concern | Recommendation | |---------|----------------| | Latency | Cascade retrieval with early stopping, cache popular queries, async rerank | | Quality | Validate offline gains via interleaving experiments and citation checks | | Safety | Source whitelisting, poisoning detection, output filtering for open retrieval | | Cost | Route to smaller models, distillation, hybrid sparse+dense retrieval |
Context
The post places this study alongside related work on generative AI traffic to retail sites, LLM-as-judge relevance evaluation, and recommendation foundation models, suggesting readers follow the chain retrieval → ranking → generation/agents → evaluation when exploring complementary literature.
> Note: This is a forum index entry; for exact quantitative findings, consult the original Semrush blog post.