Abstract
Explainable recommendation systems aim to provide users with both accurate item suggestions and natural language justifications. However, standard Transformer architectures often struggle to bridge the gap between discrete user/item IDs and linguistic tokens, frequently resulting in generic, non-personalized explanations. This paper introduces PETER (PErsonalized Transformer for Explainable Recommendation), a unified framework that utilizes a novel context prediction task and a specialized masking mechanism. PETER allows a compact, unpretrained model to achieve superior performance in both recommendation accuracy and explanation quality, significantly outperforming fine-tuned large-scale models like BERT while maintaining high training efficiency.
1. Introduction
In modern recommender systems, explainability is crucial for building user trust and improving the overall user experience. Providing a personalized explanation, such as "the style of this jacket is fashionable," helps users understand the rationale behind a recommendation. Despite the success of Transformers in natural language processing, their application to explainable recommendation faces two primary hurdles:
- The ID Gap: User and Item IDs, which are essential for personalization, reside in a different semantic space than natural language tokens.
- The Frequency Problem: Because IDs appear much less frequently than common words, models often ignore these "rare" tokens, relying instead on generic start tokens (e.g.,
<bos>). This leads to the generation of "identical explanations" that lack user-specific relevance.
2. Methodology
The proposed PETER framework employs a 2-layer Transformer architecture designed for multi-task learning. It simultaneously addresses rating prediction, context prediction, and explanation generation.
2.1 PETER Masking
Standard Transformers typically use a strict left-to-right causal mask. In contrast, PETER introduces a specialized masking matrix that allows User ($u$) and Item ($i$) ID tokens to attend to one another. This bidirectional interaction ensures that the initial representations capture the specific user-item relationship before the text generation process begins.
2.2 Context Prediction Task
A key innovation of PETER is the "context prediction" task. The model is trained to predict the set of words (features) likely to appear in the final explanation based solely on the User and Item IDs. This task acts as a "drafting" phase, forcing the model to align ID embeddings with linguistic concepts, thereby bridging the semantic gap.
2.3 Multi-Task Learning
The model optimizes a joint loss function comprising three components:
- Recommendation: Predicting the numerical rating using Mean Squared Error (MSE).
- Context Prediction: Identifying the relevant content tokens using Negative Log-Likelihood (NLL).
- Explanation Generation: Sequencing tokens into a coherent sentence using NLL.
3. Experimental Evaluation
PETER was evaluated on three large-scale datasets: Yelp (~1.3M records), Amazon Movies & TV (~441k records), and TripAdvisor Hotels (~320k records). Each dataset provides user-item interactions, ratings, and ground-truth explanations extracted from reviews.
4. Results and Discussion
Experimental results demonstrate PETER's effectiveness across several dimensions:
- Text Quality: PETER+ (the feature-enhanced version) significantly outperformed baselines, including fine-tuned BERT (ACMLM). On the Yelp dataset, PETER+ achieved a BLEU-4 score of 3.43, compared to just 0.24 for BERT.
- Personalization: PETER achieved a high Unique Sentence Ratio (USR), indicating its ability to generate diverse, user-specific explanations rather than generic templates.
- Efficiency: Due to its compact 2-layer architecture, PETER+ is highly efficient. On TripAdvisor, it trained 14 times faster than BERT-based models (2.3 minutes vs. 32.3 minutes per epoch).
- Recommendation Accuracy: The model maintained recommendation performance comparable to established algorithms like SVD++ and PMF, showing that the explanation task does not compromise the primary goal of accurate prediction.
5. Conclusion
PETER demonstrates that task-specific architecture design can be more effective than massive general-purpose pre-training for specialized tasks like personalized recommendation. By treating IDs as "out-of-vocabulary" tokens and using context prediction to force alignment, PETER generates high-quality, personalized explanations efficiently. Future work includes exploring multimodal explanations and extending the framework to cross-lingual recommendation scenarios.
Summary updated