Abstract
Sequential recommendation systems often struggle with the "black box" nature of Large Language Models (LLMs) and the opacity of traditional ID-based embeddings. This paper introduces Guided Embedding Refinement, a method that leverages LLMs to generate interpretable, domain-specific attribute scores. By fusing these "guided" embeddings with standard recommendation vectors, the proposed approach achieves significant performance gains (10% to 50%) while enhancing system transparency and maintaining inference efficiency.
Introduction
Sequential recommendation systems—the technology predicting future user actions based on historical data—traditionally rely on "ID-based embeddings." While mathematically efficient, these vectors are opaque to humans. Large Language Models (LLMs) have recently been integrated to provide semantic depth, but they introduce significant challenges: high inference latency, lack of transparency, and poor generalization across diverse tasks. This research proposes a workaround that uses LLMs as auxiliary "expert consultants" to refine traditional embeddings with human-readable logic.
Methodology
The authors introduce a two-stage process designed to bridge the gap between latent ID spaces and semantic understanding.
1. Guided Embedding Generation
Instead of tasking an LLM with direct recommendation, the model is fine-tuned to act as a critic. Using LoRA (Low-Rank Adaptation), a small, efficient LLM (such as Llama 3.2 1B) is trained to score items and users on specific, interpretable attributes. For instance, in a movie context, the LLM might evaluate "story complexity" or "emotional intensity."
2. Embedding Refinement
In the second stage, these attribute scores (the "guided" embeddings) are normalized and concatenated with a reduced-dimension version of a standard "base" embedding (e.g., from SASRec or BERT4Rec). Crucially, the LLM-generated scores remain frozen during the training of the final recommendation module. This ensures that the interpretable meaning of the attributes is preserved and not "drifted" by the optimization process.
Experimental Setup
The framework was evaluated using the Amazon Reviews 2023 dataset across three distinct categories to test versatility:
- Movies and TV: Characterized by high semantic complexity.
- Clothing, Shoes, and Jewelry: Driven by style and occasion.
- Video Games: Driven by mechanics and graphics.
Key Insights and Results
- Performance Gains: The "Refined" models consistently outperformed "Base" models, showing improvements of 10% to 50% across metrics such as Mean Reciprocal Rank (MRR) and Recall@k. The gains were most pronounced when paired with Transformer-based architectures like SASRec.
- Semantic Quality vs. Quantity: The researchers found that simply increasing the dimensionality of traditional embeddings did not match the performance boost provided by adding guided semantic information. The quality of the information in the vector proved more critical than the number of dimensions.
- Interpretability via Alignment: By inspecting the guided scores, auditors can understand why a recommendation was made. If a user is recommended a film because both the user profile and the item score highly on "intellectual challenge," the system's logic becomes transparent.
- Efficiency: Because the LLM is used only to generate embeddings (which can be done offline), the real-time recommendation speed remains identical to traditional, non-LLM systems.
Conclusion
Guided Embedding Refinement offers a practical path toward more interpretable and performant recommendation systems. By emulating the logic of a human salesperson—evaluating specific needs and styles—the framework provides the benefits of LLM-scale understanding without the associated latency or opacity of "black box" generative models. Future work will explore automated attribute selection and scaling to industrial-sized datasets.
Summary updated