Scaling the Instagram Explore Recommendations System (Meta Engineering, Aug 2023)
Overview
Instagram Explore helps users discover photos and videos from accounts they do not already follow, making it one of the largest-scale recommendation surfaces at Meta. This engineering blog post, published August 9, 2023, explains how the team scaled the Explore recommendations system to handle billions of pieces of content and serve ranked results in real time, while balancing recommendation quality against compute cost.
System Architecture
The Explore ranking pipeline is organized in stages, following the classic funnel used in large-scale recommenders:
1. Candidate generation / retrieval — A media retrieval layer, often called the IG Query system, uses embeddings and nearest-neighbor search over a corpus of billions of media items to assemble a candidate set for each ranking session. 2. First pass: value model (VM) — A lightweight model predicts the probability of multiple engagement types (e.g., likes, saves, shares, comments) and computes a utility score per candidate. This stage filters the large candidate pool cheaply. 3. Second pass: multi-task tower model (MTTM) — A heavier model uses a shared encoder (trunk) with task-specific towers to score the surviving candidates in finer detail, allowing more expensive features and computation per candidate. 4. Final ranking — Scores are combined with heuristics, rules, and diversity logic to produce the final ranked feed.
Key Engineering Contributions
- Moving work to where it is cheapest. The team shifted expensive per-candidate feature computation out of the lighter stage so that each stage spends compute only on candidates likely to survive it, reducing total CPU load without hurting ranking quality.
- Consolidation of lightweight filtering steps to cut redundant per-candidate work in the first ranking pass.
- Multi-task tower architecture that shares computation across many engagement objectives via a common trunk, improving both efficiency and consistency of predictions.
- System-level tuning of model serving, feature computation, and load distribution across services to improve utilization at Instagram-scale traffic.
- Per-surface tuning of model capacity and features, since compute/quality trade-offs differ between Explore and other Instagram surfaces.
- Meta AI blog and engineering blog posts on Instagram ranking and recommendation infrastructure
- Related entries in this collection: Foundation Model for Personalized Recommendation by Netflix (Mar 2025), and other industrial recommender system case studies
Results
According to the post, these changes improved recommendation quality and engagement metrics on Explore while lowering compute costs and infrastructure load, demonstrating that careful allocation of work across the ranking funnel can deliver both quality and efficiency gains at very large scale.
Takeaways for Practitioners
1. A staged funnel (cheap retrieval → light ranker → heavy ranker) remains the dominant industrial pattern for billion-scale recommendation. 2. Efficiency gains often come from system design — where features are computed and how stages are consolidated — as much as from model architecture. 3. Shared-trunk, multi-task models let a single heavy pass serve many objectives without proportional cost growth. 4. Quality/cost trade-offs must be tuned per product surface rather than globally.
Related Reading
Glossary
| Term | Meaning | |------|---------| | Candidate generation | Retrieval stage that selects a manageable subset of items from a huge corpus | | Value model (VM) | Lightweight first-pass ranker predicting engagement probabilities | | MTTM | Multi-task tower model; shared trunk with per-task heads for the heavy ranking pass | | ANN search | Approximate nearest-neighbor search over embeddings for fast retrieval |