Transformers4Rec: Bridging the Gap Between NLP and Sequential/Session-Based Recommendation (RecSys 2021)

Overview

This post indexes the RecSys 2021 paper Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based Recommendation, filed under the *Sequential Recommendation* section of the reading list.

| Field | Content | |-------|---------| | Title | Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based Recommendation | | Venue | RecSys 2021 | | Source | https://dl.acm.org/doi/10.1145/3460231.3474255 | | Type | Academic paper | | Section | Sequential Recommendation |

Background and Motivation

In large-scale search, recommendation, and personalization systems, sequential recommendation has long faced challenges in efficiency, scalability, and understanding user intent. Transformers4Rec addresses this by transferring transformer architectures that succeeded in NLP—where models like BERT, GPT, and XLNet revolutionized sequence modeling—into session-based and sequential recommendation, treating user interaction histories as sequences analogous to token sequences in language.

Position in the Field

The post situates the paper along the evolution of neural recommendation:

• Matrix factorization → deep CTR models → sequential Transformers → LLM-based instruction-following and generative recommendation (Gen-Rec).
• Core tensions remain: sparse user behavior, enormous item catalogs, and multi-objective business trade-offs.
• Transformer-based sequence models provide strong temporal modeling of user intent, while modern LLM-based methods add semantic priors and cold-start capability at the cost of inference expense and hallucination risk.

Evaluation Context

Typical evaluation protocols in this line of work include:

• Datasets: public recommendation corpora and session logs.
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, plus latency and serving cost considerations.
• Baselines: classical sequential recommenders and bidirectional encoders such as BERT4Rec.

For exact quantitative results, readers should consult the original PDF via the DOI link above.

Related Entries

• BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations
• EAGER: Two-Stream Generative Recommender
• Efficient On-Device Session-Based Recommendation (ACM TORS)
• How to Index Item IDs for Recommendation Foundation Models (P5, SIGIR)
• LLMCDSR: Enhancing Cross-Domain Sequential Recommendation with LLMs
• Mamba4Rec: Efficient Sequential Recommendation with Selective State Space Models

Engineering Takeaways

Practical considerations highlighted for deploying sequence-based recommenders:

1. Latency: p99 budgets, cascade design, early stopping, caching of popular queries. 2. Quality: ensure offline metric gains translate into online CTR/satisfaction via interleaving experiments. 3. Data: session/click logs remain critical; guard against distribution shift and leakage. 4. Safety and cost: source filtering, output validation, model routing, and distillation to control per-request cost.

Reference

• Original paper: *Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based Recommendation*, RecSys 2021. https://dl.acm.org/doi/10.1145/3460231.3474255