Representation Learning with Large Language Models for Recommendation (WWW 2024)
This post is a curated entry for the paper "Representation Learning with Large Language Models for Recommendation", published at WWW 2024.
- Source: https://dl.acm.org/doi/abs/10.1145/3589334.3645458
- Venue: The Web Conference (WWW) 2024
- Area: Recommender systems × Large Language Models
- A unified perspective for organizing scattered related work in LLM-based recommendation.
- A decomposition of method components: representation learning, retrievers, rerankers, planners, generators, and feedback mechanisms.
- Discussion of interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration.
- Open problems: evaluation credibility, latency and cost, hallucination and safety, cross-lingual and multimodal extension.
- Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, public recommendation benchmarks.
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency and token cost.
- Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs.
- IR — Information Retrieval
- RAG — Retrieval-Augmented Generation
- LTR — Learning to Rank
- nDCG — Normalized Discounted Cumulative Gain
- Agentic Search — search modeled as sequential decision-making with tool calls
- Gen-IR — Generative Information Retrieval
- 360Brew: A Decoder-only Foundation Model for Personalized Ranking
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers
- Augmenting Netflix Search with In-Session Adapted Recommendations
- Bridging Language and Items for Retrieval and Recommendation
- Data-efficient Fine-tuning for LLM-based Recommendation, SIGIR 2024
- DiffKG: Knowledge Graph Diffusion Model for Recommendation, WSDM 2024
> Note: The original forum post is largely a template-based summary. Quantitative results should be verified against the paper PDF.
Context and Motivation
Large-scale search and recommendation systems have long faced challenges in efficiency, scalability, and user-intent understanding. Traditional pipelines split retrieval, ranking, and generation into disjoint stages, which makes it hard to meet LLM-era expectations for natural-language interaction, multi-hop reasoning, and up-to-date knowledge. This paper addresses the intersection of representation learning with LLMs and recommendation, aiming to advance both theory and practice in this cross-domain area.
Core Themes
Typical Method Pipeline
1. Input & representation — encode queries, documents, and user context as dense/sparse representations or structured prompts. 2. Core modules — retriever, reranker, planner, memory, tool interfaces, composed in series or parallel. 3. Learning strategies — supervised fine-tuning, contrastive learning, distillation, RL (including process rewards), synthetic data bootstrapping. 4. Inference strategies — single-pass retrieval, iterative retrieval, parallel sub-queries, early stopping and budget control.
Evaluation Landscape
Takeaways for Search / Rec / Personalization
1. Architecture: cascade retrieval + rerank + generate remains mainstream, but agentic paradigms make "how many retrievals and what policy" itself learnable. 2. Data: high-quality instruction data and click/session logs matter equally; synthetic data must guard against leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Product: latency, cost, explainability, and safety are hard constraints for industrial deployment—not just academic benchmarks.
Engineering Checklist
| Item | Question | Suggestion | |------|----------|------------| | Data | PII in training/index? Versioning? | Partitioned indexes, anonymization, rollback-able embedding versions | | Latency | p99 budget? Retrieval steps? | Cascade + early stop, hot-query caching, async reranking | | Quality | Do offline gains transfer to online CTR/satisfaction? | Interleaving experiments, human audits, citation checks | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelists, adversarial detection, output filtering | | Cost | Per-query tokens and GPU usage? | Small-model routing, distillation, hybrid sparse+dense |