Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond

Qi Wang, Jindong Li, Shiqi Wang, Qianli Xing, Runliang Niu, He Kong, Rui Li, Guodong Long, Yi Chang, Chengqi ZhangarXiv 2024 · DOI · OpenReview

TL;DR

Large Language Models (LLMs) are fundamentally reshaping recommender systems (RS), evolving them from simple ID-based correlation engines into semantically aware assistants capable of complex reasoning. This survey introduces a novel three-tier taxonomy—Representing, Scheming, and Deploying—to bridge the gap between academic research and large-scale industrial application.

Bridging the Semantic Gap in Recommendation

Traditional recommender systems are often "semantic-lite." They rely heavily on collaborative filtering and user/item IDs to learn latent embeddings, frequently overlooking the rich context of human language and common-sense reasoning. While LLMs offer a promising solution by injecting deep semantic understanding, the research landscape has remained fragmented, often viewed through a narrow NLP-centric lens.

This paper shifts the focus back to the recommender system community. It specifically addresses the "industrial gap"—the significant challenge of deploying computationally intensive LLMs within high-traffic, real-time environments like those managed by Amazon, Alibaba, or TikTok.

A Three-Tier Taxonomy for LLM-based RS

To organize the diverse advancements in this field, the authors propose a three-tier taxonomy that mirrors the lifecycle of a modern recommender system.

The three-tier taxonomy: Representing, Scheming, and Deploying.
  1. Representing and Understanding: This foundational tier examines how LLMs ingest and process diverse data types. It encompasses Uni-Modality (text and IDs) and Multi-Modality (images, video, audio). A key focus here is "Understanding" through explanations—both Pre-Recommendation (why an item is being considered) and Post-Recommendation (justifying the final selection to the user).
  2. Scheming and Utilizing: This tier describes the integration strategies.
  3. Non-Generative: LLMs are used to enhance traditional ranking or scoring models, often as sophisticated feature extractors.
  4. Generative: Recommendation is treated as a natural language task where the model directly generates the recommended item or list.
  5. Retraining vs. Reusing: This distinguishes between models that undergo fine-tuning (e.g., Naive Fine-Tuning, LoRA) and those utilized via zero-shot or few-shot techniques (e.g., Prompt Tuning, In-Context Learning).
  6. Industrial Deploying: This practical tier addresses real-world constraints, including Scalability, Acceleration (e.g., speculative decoding), handling Cold Start scenarios, and implementing Dynamic Updates to keep pace with rapidly shifting trends.

The LLM-based Recommendation Pipeline

The transition from raw input data to a final industrial deployment involves a sophisticated pipeline that leverages the strengths of LLMs at various stages.

End-to-end pipeline of LLM-based recommender systems across the three tiers.

The pipeline illustrates how input data (user-item interactions, text, images, etc.) is transformed into embeddings or prompt templates. These are then processed through either non-generative or generative LLM schemes. Finally, the system undergoes rigorous industrial validation, including A/B testing and efficiency optimizations, to ensure it meets the demands of platforms like Taobao or JD.com.

Key Insights and Industrial Realities

  • From IDs to Intent: The core shift is moving from treating a user as an anonymous ID (e.g., "User_8832") to a rich profile (e.g., "a 30-year-old engineer who enjoys sci-fi"). LLMs allow systems to understand user intent and context rather than just historical correlation.
  • The Efficiency Hurdle: While generative recommendation is conceptually elegant, its latency is often prohibitive for real-time ranking. Currently, industrial success stories often involve using LLMs as "feature enhancers" or "knowledge graph builders" that support, rather than replace, the existing ranking pipeline.
  • Speculative Decoding: A notable technical crossover is the use of "speculative decoding." In this setup, a smaller, faster model drafts potential recommendations, which are then verified or refined by a larger LLM, significantly reducing latency in production.

Quantitative Breakthroughs

The survey highlights several key results from recent literature:

  • Acceleration: The DARE framework achieved a 3.45x speedup in online advertising scenarios without sacrificing performance.
  • Scaling Laws: Research from Meta (HSTU) confirms that recommendation models follow scaling laws similar to LLMs; increasing model size and data volume consistently improves performance.
  • Generative Efficiency: By redefining recommendation as a sequential transduction task, Generative Recommenders (GRs) have shown they can reduce model complexity while maintaining high accuracy.

Future Challenges

Despite the progress, several hurdles remain:

  • Latency-Cost Tradeoff: Balancing the reasoning power of a 70B parameter model with millisecond response requirements is an ongoing struggle.
  • Evaluation Metrics: Traditional metrics like Recall and NDCG may fail to capture the nuanced value of an LLM’s reasoning or the quality of its natural language explanations.
  • Privacy and Ethics: LLMs are prone to memorizing training data, which raises significant privacy concerns when dealing with sensitive user interaction history.
  • Dynamic Adaptation: Recommender systems must update within minutes to reflect new trends, yet retraining large models is time-consuming. This necessitates advancements in "unlearning" and rapid-update techniques.

Summary updated

Abstract

Large language models (LLMs) have not only revolutionized the field of natural language processing (NLP) but also have the potential to bring a paradigm shift in many other fields due to their remarkable abilities of language understanding, as well as impressive generalization capabilities and reasoning skills. As a result, recent studies have actively attempted to harness the power of LLMs to improve recommender systems, and it is imperative to thoroughly review the recent advances and challenges of LLM-based recommender systems. Unlike existing work, this survey does not merely analyze the classifications of LLM-based recommendation systems according to the technical framework of LLMs. Instead, it investigates how LLMs can better serve recommendation tasks from the perspective of the recommender system community, thus enhancing the integration of large language models into the research of recommender system and its practical application. In addition, the long-standing gap between academic research and industrial applications related to recommender systems has not been well discussed, especially in the era of large language models. In this review, we introduce a novel taxonomy that originates from the intrinsic essence of recommendation, delving into the application of large language model-based recommendation systems and their industrial implementation. Specifically, we propose a three-tier structure that more accurately reflects the developmental progression of recommendation systems from research to practical implementation, including representing and understanding, scheming and utilizing, and industrial deployment. Furthermore, we discuss critical challenges and opportunities in this emerging field. A more up-to-date version of the papers is maintained at: https://github.com/jindongli-Ai/Next-Generation-LLM-based-Recommender-Systems-Survey.
BibTeX
@misc{wang2024towards,
  author = {Qi Wang and Jindong Li and Shiqi Wang and Qianli Xing and Runliang Niu and He Kong and Rui Li and Guodong Long and Yi Chang and Chengqi Zhang},
  title = {{Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond}},
  year = {2024},
  eprint = {2410.19744},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.19744},
}
Figure