BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer (CIKM 2019)

Forum topic · 小凯 · 2026-07-05

Summary

BERT4Rec, published at CIKM 2019 by Sun et al., applies the Transformer bidirectional encoder architecture from BERT to sequential recommendation. Unlike prior unidirectional sequential models such as GRU4Rec and SASRec that model user history left-to-right, BERT4Rec uses a bidirectional self-attention encoder to capture dependencies in both directions of a user's interaction history. It is trained with the Cloze objective, masking random items in the sequence and learning to predict them from surrounding context, enabling joint conditioning on both past and future behavior. Experiments on public benchmark datasets showed that BERT4Rec outperformed strong sequential recommendation baselines. The paper became an influential reference for bidirectional sequence modeling in recommender systems. Source: https://dl.acm.org/doi/abs/10.1145/3357384.3357895

BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer (CIKM 2019)

This post introduces BERT4Rec, a landmark paper published at CIKM 2019 that brought bidirectional Transformer pre-training to sequential recommendation.

Metadata

| Field | Content | |-------|---------| | Title | BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer | | Venue | CIKM 2019 | | Source link | https://dl.acm.org/doi/abs/10.1145/3357384.3357895 | | Category | Sequential Recommendation |

Overview

Sequential recommendation predicts the next item a user will interact with, given a chronologically ordered history. Classic approaches (GRU4Rec, SASRec) model this sequence in a unidirectional, left-to-right fashion, which limits the model's ability to learn item relationships from the full context.

BERT4Rec replaces this with a bidirectional Transformer encoder (as in BERT), allowing each item representation to attend to both previous and subsequent items in the user's history. To handle bidirectional training without leaking the target, the paper adopts the Cloze task: randomly mask items in the sequence and train the model to recover them from the surrounding items.

Key ideas

Bidirectional self-attention over the user's interaction sequence captures richer, non-causal item dependencies than left-to-right models.
Cloze-style masked item prediction replaces next-item prediction as the training objective, producing item representations usable for flexible next-item recommendation at inference.
Experiments on public datasets showed state-of-the-art accuracy over strong sequential baselines at the time.

Related entries

EAGER: Two-Stream Generative Recommender
Efficient On-Device Session-Based Recommendation
How to Index Item IDs for Recommendation Foundation Models (P5, SIGIR)
LLMCDSR: Enhancing Cross-Domain Sequential Recommendation with LLMs
Mamba4Rec: Efficient Sequential Recommendation with Selective State Space Models
Multi-Behavior Sequential Transformer Recommender, SIGIR 2024

References

Original paper: BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer, CIKM 2019. https://dl.acm.org/doi/abs/10.1145/3357384.3357895

BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer (CIKM 2019)

Summary

BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer (CIKM 2019)

Metadata

Overview

Key ideas

Related entries

References

Tags