TL;DR
AdaptRec is a novel framework that enhances Large Language Model (LLM) recommendation systems by teaching them to "think" like collaborative filters. By using a self-adaptive mechanism where the LLM selects its own "similar user" demonstrations to include in its prompt, the system achieves double-digit performance gains in predicting a user's next move.
What Problem Does This Paper Address?
Sequential recommendation systems—the engines that predict what movie you’ll watch or song you’ll hear next—traditionally rely on two things: your personal history and "collaborative signals" (what people with similar tastes did).
While LLMs are fantastic at understanding the semantic context of item titles (e.g., knowing that The Hobbit and Lord of the Rings are related), they struggle to incorporate collaborative signals effectively. Current "LLM-for-Rec" approaches usually either focus only on a single user's history or use rigid, mathematical similarity scores (like Euclidean distance) to fetch similar users. These "static" methods often fail because a mathematical similarity in a high-dimensional embedding space doesn't always translate into a reasoning pattern that an LLM can use to make a better prediction.
The authors identify three main hurdles:
- Quality Control: How do we know if a "similar" user's history is actually a helpful example for the LLM?
- Scale: How do we find these users in a dataset of millions without melting the server?
- Adaptability: How can the selection process improve as the model learns?
Methods
The authors propose AdaptRec, a three-stage pipeline that bridges the gap between traditional collaborative filtering and LLM reasoning:
- User Similarity Retrieval (Coarse-grained): To handle the massive search space, they use standard vector embeddings and cosine similarity to grab a "Top-N" pool of potentially similar users. This is the "fast and cheap" filter.
- Self-Adaptive User Selection (Fine-grained): This is the core innovation. They use a specialized User-Based Similarity Retrieval Prompt to ask the LLM itself to rank the candidates from Stage 1. During training, the LLM learns to identify which users provide the most "informative" signals for the final task.
- Contextual Prompt-based Recommendation: The histories of the top-ranked similar users are converted into natural language "demonstrations" (e.g., "Similar user 1 watched X, then Y, then chose Z"). These are fed into the final prompt alongside the target user's history to guide the LLM's prediction.
The model is fine-tuned using LoRA (Low-Rank Adaptation), which allows for efficient training by only updating a small fraction of the LLM's parameters.
Datasets
The framework was evaluated on three diverse, real-world datasets:
- MovieLens (100k): A classic benchmark for movie recommendations.
- LastFM: Music artist listening histories.
- GoodReads: Book interaction data, specifically focused on the "History" genre.
The authors used a standard 8:1:1 split for training, validation, and testing, employing a "leave-one-out" strategy for evaluation.
Key Insights and Ideas
- The LLM as a Curator: Instead of treating the LLM as a passive consumer of data, AdaptRec makes the LLM an active participant in selecting its own training context. This "self-adaptive" loop ensures the demonstrations align with the model's internal reasoning process.
- The "Sweet Spot" of Context: More data isn't always better. The researchers found that performance peaks at 5 demonstrations. Adding more (7 or 9) actually degraded performance, likely because the prompt became too noisy or exceeded the model's effective attention span for the task.
- Beyond Genre Matching: In a qualitative case study, the authors showed that a standard prompt might recommend a movie just because it's in the same genre (e.g., recommending a romance movie to a romance fan). However, AdaptRec's collaborative signals allowed it to identify non-obvious "cross-genre" patterns (e.g., realizing that fans of certain romantic dramas also frequently watch Lord of the Rings), leading to more accurate, data-driven predictions.
Key Results
AdaptRec significantly outperformed both traditional sequential models (like SASRec) and state-of-the-art LLM-based recommenders (like LLaRA):
- HitRatio@1 (Accuracy): Improved by 7.13% on MovieLens, 18.16% on LastFM, and 10.41% on GoodReads compared to the next-best method.
- Few-Shot Prowess: In scenarios with very little training data, the gains were even more dramatic—up to 23% improvement. This suggests the prompt design is exceptionally good at "teaching" the model the task without requiring massive retraining.
- Faster Convergence: The inclusion of collaborative demonstrations helped the model converge (reach its peak performance) much faster during training than models relying on user history alone.
Limitations and Future Work
- Multilingual Challenges: The model showed slightly lower gains on MovieLens compared to LastFM. The authors attribute this to MovieLens containing multilingual titles, which can be harder for English-centric LLMs to process semantically.
- Computational Cost: The two-phase selection process and iterative feedback loop add computational overhead compared to simpler, static recommendation models.
- Future Directions: The authors plan to explore multi-modal signals—incorporating book covers or movie posters—to see if visual data can further refine the "similarity" of users. They also aim to optimize the framework for "ultra-large-scale" production environments.
Summary updated