Plug-In Diffusion Model for Sequential Recommendation (AAAI 2024)
Overview
This paper, published at AAAI 2024, proposes a plug-in diffusion model for sequential recommendation. The key idea is not to build a standalone generative recommender, but to attach a pretrained diffusion model on top of an existing sequential recommendation model as a plug-in enhancement module.
Background and Motivation
Sequential recommendation predicts a user's next interaction based on their historical behavior sequence. Mainstream approaches (e.g., GRU4Rec, SASRec, BERT4Rec) encode behavior sequences and score candidate items, but they commonly suffer from:
- Noisy or ambiguous behavioral signals, where clicked items do not necessarily reflect true intent;
- Imprecise top-K ranking, especially for items just outside the user's immediate affinity boundary;
- Limited ability to model the *distribution* over plausible next items rather than a point estimate.
- A plug-in architecture that treats a pretrained diffusion model as an enhancement module for existing sequential recommenders.
- A diffusion-based mechanism that denoises and augments the top-K candidates using guidance from user representations.
- Empirical results on standard sequential recommendation benchmarks showing improvements over strong baselines, with the plug-in design transferring across different base models.
- Generative recommenders (e.g., EAGER, P5/item-ID indexing work) treat generation itself as the recommendation mechanism.
- Sequence encoders (BERT4Rec, Mamba4Rec) improve behavioral sequence modeling efficiency and accuracy.
- This paper instead shows that diffusion models can serve as a complementary refinement stage, bridging discriminative sequential models and generative modeling, with practical benefits of reusability and low integration cost.
- BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations
- EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration
- Mamba4Rec: Efficient Sequential Recommendation with Selective State Space Models
- How to Index Item IDs for Recommendation Foundation Models (P5, SIGIR)
- Original paper: https://ojs.aaai.org/index.php/AAAI/article/view/28736
Diffusion models have shown strong capability in generative modeling by learning to iteratively denoise data. The authors leverage this to refine the item distribution implied by a base recommender.
Method
The proposed framework operates as a plug-in stage after a base sequential recommender:
1. Base recommender: an off-the-shelf sequential model produces user representations and a set of top-K candidate items. 2. Diffusion-based augmentation: a diffusion model, conditioned on the user representation and behavior sequence, refines/augments the top-K items by sampling from a denoised distribution over items, guided by the user's intent. 3. Final ranking: the enhanced item set improves the final top-K recommendation quality.
Because the diffusion module is trained to complement rather than replace the base model, the approach is model-agnostic and can be combined with different underlying recommenders.
Contributions
Position in the Field
This work sits within the growing line of research applying generative models to recommendation: