Pre-train, Prompt, and Recommendation: A Survey of Language Modeling Paradigm Adaptations in Recommender Systems

Forum topic · 小凯 · 2026-07-05

Summary

This forum post indexes the survey 'Pre-train, Prompt, and Recommendation: A Comprehensive Survey of Language Modeling Paradigm Adaptations in Recommender Systems', published in TACL (MIT Press) in December 2023. The survey systematically reviews how language modeling paradigms—pre-training, prompting, and LLM-based generation—are adapted for recommender systems. The post organizes the field into a taxonomy covering modeling paradigms (discriminative vs. generative retrieval), LLM integration (RAG, agents, tool use), optimization objectives, and evaluation protocols. It traces the field's evolution from BERT-based reranking and dense retrieval (2019–2021) through retrieval-augmented generation (2022–2023) to conversational and agentic search and generative recommendation systems (2024 onward). Key open problems highlighted include benchmark standardization, LLM evaluation bias, hallucination and safety, latency and cost constraints, and cross-lingual/multimodal generalization. The post also provides an engineering checklist covering data privacy, p99 latency budgets, offline-to-online metric alignment, retrieval poisoning defenses, and per-query token costs, plus cross-references to related surveys on LLM-powered agents, retrieval methods, and generative recommendation.

Pre-train, Prompt, and Recommendation: A Comprehensive Survey of Language Modeling Paradigm Adaptations in Recommender Systems

Source: TACL, MIT Press, Dec 2023 | Type: Survey

Overview

This survey examines how language modeling paradigms—pre-training, prompting, and large language model (LLM) generation—have been adapted for recommender systems. Traditional pipelines separate retrieval, ranking, and generation, but LLM-era systems demand natural language interaction, multi-hop reasoning, and real-time knowledge integration.

Key Contributions

A unified perspective that organizes scattered related work into a comparable framework
Clear decomposition of method components: representation learning, retrievers, rerankers, planners, generators, and feedback mechanisms
Reproducible benchmarks, datasets, and taxonomy tables lowering entry barriers for researchers
Discussion of interfaces with LLM tool calling, reinforcement learning, and multi-agent collaboration
Explicit open problems: evaluation trustworthiness, latency/cost, hallucination and safety, cross-lingual and multimodal extension

Taxonomy

| Dimension | Subcategories | Representative Ideas | Strengths | Limitations | |---|---|---|---|---| | Modeling paradigm | Discriminative / generative retrieval | Two-tower, cross-encoder, DSI | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool-use | Retrieval augmentation, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Scarce annotations | | Evaluation | Offline / online / human | nDCG, MRR, LLM-as-judge, A/B | Comparable | Gap with real satisfaction |

Research Timeline

2019–2021: BERT reranking and DPR establish neural retrieval foundations
2022–2023: RAG drives retrieval-generation fusion
2024+: Conversational/agentic search and Gen-RecSys proliferate
2025–2026: RL-trained search agents, Deep Research, and GraphRAG become growth frontiers

Evaluation Paradigms

Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, public recommendation sets
Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, token cost
Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs

Insights for Search / Rec / Personalization

1. Architecture: Cascaded retrieval + rerank + generation remains mainstream, but agentic paradigms make retrieval strategy itself learnable 2. Data: High-quality instruction data and click/session logs are both critical; synthetic data risks leakage and distribution shift 3. Evaluation: The offline/online gap is widening; LLM-as-judge needs cross-validation with human evaluation 4. Production: Latency, cost, interpretability, and safety are hard constraints beyond academic benchmarks

Engineering Checklist

| Item | Question | Recommendation | |---|---|---| | Data | PII in training/index? Version management? | Partitioned indexes, anonymization, rollbackable embeddings | | Latency | p99 budget? Retrieval steps? | Cascade + early stop, query caching, async rerank | | Quality | Do offline gains translate to online CTR? | Interleaving experiments, human audits, citation checks | | Safety | Poisoning/bias from open retrieval? | Source whitelists, adversarial detection, output filtering | | Cost | Per-query token and GPU usage? | Small-model routing, distillation, hybrid sparse+dense |

Open Problems

Limited unified benchmarks, non-reproducible private data, LLM evaluation bias, and safety/cost constraints of agentic systems. Future directions include finer-grained process supervision, retrieval-reasoning co-training, enterprise metadata governance, and multimodal/cross-lingual consistency.

Related Entries

A Comprehensive Review on Harnessing Large Language Models (arXiv 2507.21117)
A Comprehensive Survey on Cross-Domain Recommendation (arXiv 2503.14110)
A Comprehensive Survey on Retrieval Methods in Recommender Systems (arXiv 2407.21022)
A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)
A Survey of Generative Search and Recommendation in the Era of Large Language Models (arXiv 2404.16924)
A Survey on LLM-powered Agents for Recommender Systems (arXiv 2502.10050)