Powering Job Search at Scale: LLM-Enhanced Query Understanding in Job Matching Systems (LinkedIn, arXiv 2025)

Overview

This entry indexes an August 2025 arXiv paper from LinkedIn:

| Field | Value | |-------|-------| | Title | Powering Job Search at Scale: LLM-Enhanced Query Understanding in Job Matching Systems | | Authors | Ping Liu, Jianqiang Shen, Qianqi Shen, Chunnan Yao, Kevin Kao, Dan Xu, et al. (12 authors) | | Date | Aug 2025 | | Link | https://arxiv.org/abs/2509.09690 | | Type | Academic paper | | Category | Query Understanding, Job |

The paper addresses query understanding for large-scale job search, where traditional pipelines that treat retrieval, ranking, and generation separately struggle to interpret free-form natural-language queries under the LLM paradigm.

Context

In large-scale search, recommendation, and personalization systems, information retrieval has long faced challenges around efficiency, scalability, and user intent understanding. The paper sits at the intersection of IR and industrial job matching, responding to the question of how to re-divide responsibilities among retrieval, ranking, generation, and tool invocation in the LLM era. New variables compared to the classic search stack are the inference budget and action space (whether to retrieve, how many times, and which tools to call).

Core Contributions (as outlined in the post)

• A unified perspective for query understanding in job matching, situating scattered prior work in a comparable framework
• Decomposition of method components: representation learning, retrievers, rerankers, planners, generators, feedback mechanisms
• Discussion of interfaces with emerging paradigms: LLM tool calling, reinforcement learning, multi-agent collaboration
• Open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension

Technical Landscape

Neural IR has evolved from BM25 → BERT cross-encoders → two-tower dense retrieval → late interaction → generative retrieval and LLM agents, each balancing an efficiency–effectiveness–maintainability triangle. Dense retrieval achieves millisecond recall via ANN search but is sensitive to domain shift and long-tail queries; cross-encoders are accurate but cannot precompute document representations; generative approaches reduce cascade errors but face index-update difficulties. On the recommendation side, LLMs provide semantic priors and cold-start capability, but online inference cost and hallucination risk demand careful system design.

Evaluation Practices Noted

• Datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency and token cost
• Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs
• Note: consult the original PDF for exact quantitative results

Engineering Deployment Checklist

| Check | Question | Suggestion | |-------|----------|------------| | Data | PII in training/index? Versioning? | Partitioned indexes, anonymization, rollback-able embedding versions | | Latency | p99 budget? Retrieval steps? | Cascade + early stopping, query caching, async reranking | | Quality | Do offline gains translate to online CTR/satisfaction? | Interleaving experiments, human audits, citation checks | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelisting, adversarial detection, output filtering | | Cost | Token and GPU cost per query? | Route to small models, distillation, hybrid sparse+dense |

Related Entries

• Aligned Query Expansion: Efficient Query Expansion for Information Retrieval
• Beyond the limitation of a single query: Train your LLM for query expansion
• Decomposing Complex Queries for Tip-of-the-tongue Retrieval (2023)
• Few-Shot Generative Conversational Query Rewriting, SIGIR 2020
• [Hierarchical query classification in e-commerce search, WWW 2024]
• Hypothetical Documents or Knowledge Leakage? Rethinking LLM-based Query Expansion

Actionable Takeaways

1. Researchers: reproduce core comparisons; check whether statistical significance and compute cost are reported. 2. Engineers: extract pluggable modules (encoders, rerankers, planners) and assess integration cost with existing stacks. 3. Product managers: focus on user-perceivable gains (latency, answer trustworthiness, multi-turn consistency) rather than offline nDCG alone.

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Search modeled as sequential decision-making and tool invocation | | Gen-IR | Generative Information Retrieval |

Reference

• Original paper: arXiv:2509.09690