360Brew: A Decoder-only Foundation Model for Personalized Ranking and Recommendation

Paper: https://arxiv.org/abs/2501.16450 (arXiv, January 2025)

Authors: Hamed Firooz, Maziar Sanjabi, Adrian Englhardt, Aman Gupta, Ben Levine, Dre Olgiati, et al. (23 authors in total)

One-line Summary

360Brew presents a decoder-only foundation model for personalized ranking and recommendation, aiming to unify retrieval, ranking, and generation in large-scale search and recommendation systems.

Background and Motivation

In large-scale search, recommendation, and personalization systems, recommendation has long faced challenges around efficiency, scalability, and user-intent understanding. Traditional pipeline-based approaches tend to treat retrieval, ranking, and generation as separate stages, making it hard to meet the combined demands of the LLM era: natural-language interaction, multi-hop reasoning, and up-to-date knowledge. 360Brew was proposed in this context to push the theoretical and practical boundaries at the intersection of foundation models and recommender systems.

The problem space the paper engages with includes: open-domain information access, enterprise knowledge retrieval, conversational search, semantic understanding in recommendation, and end-to-end architectures that combine external knowledge sources with generative models.

Core Contributions

• A unified perspective for the problem domain, bringing scattered prior work into a comparable framework.
• A clear decomposition of method components (representation learning, retrievers, re-rankers, planners, generators, feedback mechanisms) to ease engineering adoption.
• Reproducible benchmarks, datasets, or taxonomy tables at the level of experimental protocol / survey coverage, lowering the entry cost for follow-up research.
• Discussion of interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration, including migration paths from research prototypes to industrial systems.
• Explicit open problems: evaluation trustworthiness, latency and cost, hallucination and safety, and cross-lingual / multimodal extension.

Method / System Architecture

Work in this space typically follows four steps: problem formulation → model/system design → training or construction pipeline → inference pipeline.

1. Input and representation: encoding queries, documents, and user context as dense or sparse representations, or structured prompts. 2. Core modules: retrievers, re-rankers, planners, memory modules, tool interfaces — chained or combined by task. 3. Learning strategies: supervised fine-tuning, contrastive learning, distillation, reinforcement learning (including process rewards), and bootstrapped data synthesis. 4. Inference strategies: single-pass retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control.

Experimental Setup and Evaluation

Typical evaluation components in this line of work include:

• Datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora, public recommendation datasets.
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, and token cost.
• Baselines: BM25, dense retrieval, cross-encoder re-ranking, retrieval-free LLMs, commercial search APIs.
• Ablations: contribution of individual modules (number of retrieval steps, re-ranking depth, training-data scale) to final quality.

Specific numerical results should be verified against the original PDF tables; this overview is based on the abstract and public metadata.

Key Takeaways for Search / Rec / Personalization

1. Architecture: cascaded retrieval + re-ranking + generation remains mainstream, but agentic paradigms are turning "how many retrieval steps and what policy" itself into a learnable object. 2. Data: high-quality instruction data and click/session logs are both critical; synthetic data requires safeguards against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation against human evaluation. 4. Product: latency, cost, explainability, and safety policies are hard constraints for industrial deployment — academic benchmarks alone are not enough.

Limitations and Future Work

Potential limitations include experiment scale constrained by GPU budgets, benchmarks that diverge from real user distributions, unknown cross-lingual generalization due to English-centric data, and safety risks of agentic systems on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal / fairness constraints for recommendation.

Cross-references

• Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers (arXiv 2402.17152)
• Augmenting Netflix Search with In-Session Adapted Recommendations (ACM, 10.1145/3523227.3547407)
• Bridging Language and Items for Retrieval and Recommendation (arXiv 2403.03952)
• Data-efficient Fine-tuning for LLM-based Recommendation, SIGIR 2024 (10.1145/3626772.3657807)
• DiffKG: Knowledge Graph Diffusion Model for Recommendation, WSDM 2024 (10.1145/3616855.3635850)
• EAGER-LLM: Enhancing Large Language Models as Recommenders (arXiv 2502.14735)

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain, a ranking quality metric | | Agentic Search | Modeling search as sequential decision-making and tool invocation | | Gen-IR | Generative Information Retrieval |