AdaptRec: A Self-Adaptive Framework for Sequential Recommendations with Large Language Models

Tong Zhang

TL;DR

AdaptRec is a novel framework that enhances Large Language Model (LLM) recommendation systems by teaching them to "think" like collaborative filters. By using a self-adaptive mechanism where the LLM selects its own "similar user" demonstrations to include in its prompt, the system achieves double-digit performance gains in predicting a user's next move.

What Problem Does This Paper Address?

Sequential recommendation systems—the engines that predict what movie you’ll watch or song you’ll hear next—traditionally rely on two things: your personal history and "collaborative signals" (what people with similar tastes did).

While LLMs are fantastic at understanding the semantic context of item titles (e.g., knowing that The Hobbit and Lord of the Rings are related), they struggle to incorporate collaborative signals effectively. Current "LLM-for-Rec" approaches usually either focus only on a single user's history or use rigid, mathematical similarity scores (like Euclidean distance) to fetch similar users. These "static" methods often fail because a mathematical similarity in a high-dimensional embedding space doesn't always translate into a reasoning pattern that an LLM can use to make a better prediction.

The authors identify three main hurdles:

  1. Quality Control: How do we know if a "similar" user's history is actually a helpful example for the LLM?
  2. Scale: How do we find these users in a dataset of millions without melting the server?
  3. Adaptability: How can the selection process improve as the model learns?

Methods

The authors propose AdaptRec, a three-stage pipeline that bridges the gap between traditional collaborative filtering and LLM reasoning:

  1. User Similarity Retrieval (Coarse-grained): To handle the massive search space, they use standard vector embeddings and cosine similarity to grab a "Top-N" pool of potentially similar users. This is the "fast and cheap" filter.
  2. Self-Adaptive User Selection (Fine-grained): This is the core innovation. They use a specialized User-Based Similarity Retrieval Prompt to ask the LLM itself to rank the candidates from Stage 1. During training, the LLM learns to identify which users provide the most "informative" signals for the final task.
  3. Contextual Prompt-based Recommendation: The histories of the top-ranked similar users are converted into natural language "demonstrations" (e.g., "Similar user 1 watched X, then Y, then chose Z"). These are fed into the final prompt alongside the target user's history to guide the LLM's prediction.

The model is fine-tuned using LoRA (Low-Rank Adaptation), which allows for efficient training by only updating a small fraction of the LLM's parameters.

Datasets

The framework was evaluated on three diverse, real-world datasets:

  • MovieLens (100k): A classic benchmark for movie recommendations.
  • LastFM: Music artist listening histories.
  • GoodReads: Book interaction data, specifically focused on the "History" genre.

The authors used a standard 8:1:1 split for training, validation, and testing, employing a "leave-one-out" strategy for evaluation.

Key Insights and Ideas

  • The LLM as a Curator: Instead of treating the LLM as a passive consumer of data, AdaptRec makes the LLM an active participant in selecting its own training context. This "self-adaptive" loop ensures the demonstrations align with the model's internal reasoning process.
  • The "Sweet Spot" of Context: More data isn't always better. The researchers found that performance peaks at 5 demonstrations. Adding more (7 or 9) actually degraded performance, likely because the prompt became too noisy or exceeded the model's effective attention span for the task.
  • Beyond Genre Matching: In a qualitative case study, the authors showed that a standard prompt might recommend a movie just because it's in the same genre (e.g., recommending a romance movie to a romance fan). However, AdaptRec's collaborative signals allowed it to identify non-obvious "cross-genre" patterns (e.g., realizing that fans of certain romantic dramas also frequently watch Lord of the Rings), leading to more accurate, data-driven predictions.

Key Results

AdaptRec significantly outperformed both traditional sequential models (like SASRec) and state-of-the-art LLM-based recommenders (like LLaRA):

  • HitRatio@1 (Accuracy): Improved by 7.13% on MovieLens, 18.16% on LastFM, and 10.41% on GoodReads compared to the next-best method.
  • Few-Shot Prowess: In scenarios with very little training data, the gains were even more dramatic—up to 23% improvement. This suggests the prompt design is exceptionally good at "teaching" the model the task without requiring massive retraining.
  • Faster Convergence: The inclusion of collaborative demonstrations helped the model converge (reach its peak performance) much faster during training than models relying on user history alone.

Limitations and Future Work

  • Multilingual Challenges: The model showed slightly lower gains on MovieLens compared to LastFM. The authors attribute this to MovieLens containing multilingual titles, which can be harder for English-centric LLMs to process semantically.
  • Computational Cost: The two-phase selection process and iterative feedback loop add computational overhead compared to simpler, static recommendation models.
  • Future Directions: The authors plan to explore multi-modal signals—incorporating book covers or movie posters—to see if visual data can further refine the "similarity" of users. They also aim to optimize the framework for "ultra-large-scale" production environments.

Summary updated

Abstract

The recent advancements in Large Language Models (LLMs) have generated considerable interest in their utilization for sequential recommendation tasks. While collaborative signals from similar users are central to recommendation modeling, effectively transforming these signals into a format that LLMs can understand and utilize remains challenging. The critical challenges include selecting relevant demonstrations from large-scale user interactions and ensuring their alignment with LLMs' reasoning process. To address these challenges, we introduce AdaptRec, a self-adaptive fram-ework that leverages LLMs for sequential recommendations by incorporating explicit collaborative signals. AdaptRec employs a two-phase user selection mechanism -- User Similarity Retrieval and Self-Adaptive User Selection -- to efficiently identify relevant user sequences in large-scale datasets from multi-metric evaluation. We also develop a User-Based Similarity Retrieval Prompt, enabling the model to actively select similar users and continuously refine its selection criteria during training. Using the collaborative signals from similar users, we construct a User-Contextualized Recommendation Prompt that translates their behavior sequences into natural language, explicitly integrating this information into the recommendation process. Experiments demonstrate AdaptRec's superior performance, with significant improvements in HitRatio@1 scores of 7.13\%, 18.16\%, and 10.41\% across real-world datasets with full fine-tuning, and even higher gains of 23.00\%, 15.97\%, and 17.98\% in few-shot scenarios.
BibTeX
@misc{zhang2025adaptrec,
  author = {Tong Zhang},
  title = {{AdaptRec: A Self-Adaptive Framework for Sequential Recommendations with Large Language Models}},
  year = {2025},
  eprint = {2504.08786},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.08786},
}

Loading related content...

Figure