Abstract
Generative recommender systems typically rely on globally-shared prompts, failing to capture individual user nuances or collaborative signals. We present PeaPOD (PersonAlized PrOmpt Distillation), a framework that personalizes generative recommendations by distilling collaborative knowledge into "soft prompts." By employing a multi-head attention mechanism to blend a user’s individual embedding with those of their most similar peers, PeaPOD creates a contextualized representation that significantly outperforms standard methods across sequential recommendation, top-n recommendation, and explanation generation tasks.
Introduction
Generative recommender systems leverage the reasoning power of Large Language Models (LLMs) to produce recommendations. These systems generally use two types of prompts: discrete text templates or learned continuous vectors (soft prompts). However, these prompts are typically "globally shared," providing identical instructions for every user. While unique prompts per user could theoretically address this, they often fail in "cold start" scenarios where interaction history is sparse.
PeaPOD addresses this by integrating collaborative knowledge. In traditional recommendation systems, users with similar interests provide critical clues for future preferences. PeaPOD bridges the gap between LLM reasoning and the collective intelligence of collaborative filtering.
Methodology
The PeaPOD workflow consists of three primary stages designed to distill personalized preferences into the LLM input:
- User Embedding Initialization: Initial embeddings are generated using standard collaborative filtering (e.g., Probabilistic Matrix Factorization), ensuring that users with similar tastes are positioned closely in the embedding space.
- Attending to Neighbors: To generate a personalized prompt, the system identifies the top-$n$ most similar users to the target. A Multi-Head Attention module is then used, where the target user's embedding serves as the Query, and the neighbor embeddings serve as the Keys and Values.
- Prompt Distillation: The output is a "collaborative user prompt"—a continuous vector capturing group preferences. This is prepended to the input of a T5 model alongside task-specific and discrete prompts to guide the generation process.
Experimental Setup
The model was evaluated on three real-world Amazon datasets: Sports, Beauty, and Toys & Games. These datasets include user/item IDs, reviews, and timestamps, providing a robust environment for testing sequential recommendation, top-n recommendation, and explanation generation.
Results and Key Insights
PeaPOD demonstrates that collaborative signals are superior to global templates for priming LLMs. Key findings include:
- Multi-Head Attention for "Facets" of Interest: Multiple attention heads allow the model to capture diverse dimensions of similarity, such as overlapping item preferences or temporal patterns.
- Performance Gains: PeaPOD-PMF outperformed baselines like P5 and POD. For example, on the Toys dataset, it achieved an HR@5 of 0.0865, surpassing the RDRec baseline (0.0723).
- Model Size Paradox: Interestingly, the smaller T5-small model often outperformed T5-base on sparse datasets, suggesting that larger models may overfit more easily when data is thin.
- ID-Only Efficiency: PeaPOD maintains strong performance using only user and item IDs, which is vital for privacy-sensitive applications where metadata is unavailable.
Conclusion and Future Work
While PeaPOD achieves state-of-the-art results in recommendation tasks, its restriction to IDs limits its performance in explanation generation compared to models using item titles. Future work will focus on integrating item metadata (titles, categories) into the attention mechanism and developing an end-to-end architecture where initial embeddings and prompt generation are trained simultaneously to further refine personalized recommendations.
Summary updated