Text Is All You Need: Learning Language Representations for Sequential Recommendation (Recformer, arXiv 2305.13731)

Forum topic · 小凯 · 2026-07-05

Summary

This forum post discusses the May 2023 arXiv paper 'Text Is All You Need: Learning Language Representations for Sequential Recommendation' (arXiv:2305.13731), authored by Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, and colleagues. The work proposes representing items purely as text so that language models can serve as the backbone of sequential recommendation. Instead of learning item ID embeddings, the approach models the next item to be recommended as a sentence, jointly learning language representations and sequential recommendation behavior. The forum entry places the paper in the Recommender Engines / Sequential Recommendation category, outlines the general pipeline of representation learning, retrieval, ranking, and generation in modern recommender systems, and cross-references related entries such as 360Brew, Netflix in-session recommendations, and LLM-based recommendation fine-tuning work. It also discusses engineering considerations including latency budgets, data privacy, evaluation reliability, and cost constraints when moving from research prototypes to production. Readers should consult the original PDF for exact experimental numbers, as the post itself is based on the abstract and public metadata.

Text Is All You Need: Learning Language Representations for Sequential Recommendation (May 2023, arXiv)

Paper: https://arxiv.org/abs/2305.13731 Authors: Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, et al. (7 authors) Category: Recommender Engines, Sequential Recommendation

Overview

This paper proposes representing items as text so that a language model can directly power sequential recommendation. Rather than relying on learned item-ID embeddings, the method treats the next item to recommend as a sentence, jointly learning language representations and sequential recommendation behavior. The core idea, as summarized by its title: text is all you need.

Context and Motivation

The post frames the work within the broader shift in large-scale search, recommendation, and personalization systems in the LLM era:

Traditional pipelines split retrieval, ranking, and generation, which limits user-intent understanding and scalability.
LLMs bring natural-language interaction, multi-hop reasoning, and semantic priors that help with cold-start and sparse user behavior.
Key tensions remain: inference latency, cost, hallucination risk, and evaluation reliability.

Method Pipeline (as described in the post)

1. Input & representation — encode queries, documents, or user context into dense/sparse representations or structured prompts. 2. Core modules — retriever, reranker, planner, memory, and tool interfaces combined in series or parallel. 3. Learning strategy — supervised fine-tuning, contrastive learning, distillation, reinforcement learning, or synthetic data bootstrapping. 4. Inference strategy — single-pass retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control.

Evaluation Notes

The post notes typical experimental setups for this line of work (datasets such as MS MARCO, BEIR, Natural Questions; metrics such as nDCG@10, MRR, Recall@k; baselines including BM25, dense retrieval, cross-encoder rerankers). It cautions that exact numbers should be verified against the original PDF, since the forum entry is compiled from the abstract and public metadata.

Key Takeaways for Practitioners

Architecture: cascaded retrieve → rerank → generate remains mainstream; agentic paradigms make retrieval strategy itself learnable.
Data: high-quality instruction data and session logs matter; beware leakage and distribution shift in synthetic data.
Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs human cross-validation.
Product: latency, cost, explainability, and safety are hard constraints for industrial deployment.

Engineering Checklist (from the post)

| Item | Question | Suggestion | |------|----------|------------| | Data | PII in training/index? versioning? | partitioned indexes, anonymization, rollback-able embeddings | | Latency | p99 budget? retrieval steps? | cascades + early stop, cache hot queries, async rerank | | Quality | offline gains → online CTR/satisfaction? | interleaving experiments, human audits, citation checks | | Safety | poisoning/bias via open retrieval? | source whitelisting, adversarial detection, output filtering | | Cost | per-query tokens and GPU usage? | small-model routing, distillation, hybrid sparse+dense |

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Search modeled as sequential decision-making with tool calls | | Gen-IR | Generative Information Retrieval |

> Original abstract: see arXiv:2305.13731.