TL;DR
This paper introduces RAP, a novel framework for sequential recommendation that tackles the challenge of noisy historical user interactions. By formalizing sequence denoising as a Markov Decision Process and employing a pattern-enhanced policy with contrastive learning, RAP effectively filters out irrelevant items to improve recommendation accuracy.
What Problem Does This Paper Address?
Sequential recommendation systems aim to predict a user's next interaction based on their past behavior. A significant challenge in this field is that not all historical interactions are equally informative. User behavior can be random, and their interests diverse, leading to "noisy" or irrelevant items in their interaction history. Including these irrelevant items can confuse recommendation models, making it difficult to accurately distill user preferences and predict future actions. Existing methods often treat all historical items equally or use attention mechanisms that might not fully remove noise. This paper addresses this gap by proposing a method to explicitly "denoise" the user's interaction sequence, focusing only on the relevant items for better preference learning.
Methods
The core of RAP lies in its two main modules:
- Pattern-enhanced Policy Module: This module frames the sequence denoising problem as a Markov Decision Process (MDP). For each item in the user's historical sequence, a policy network decides whether the item is relevant to the target item (action=1) or not (action=0). To guide this decision-making process, especially without explicit relevance labels, the system leverages sequential patterns mined from the data (using SPADE). These patterns act as prior knowledge, enriching the state representation of the policy network and helping it make more informed denoising decisions. The policy then splits the original sequence into a "positive" (relevant) and a "negative" (irrelevant) subsequence.
- Contrastive Learning Module: After obtaining the positive and negative subsequences, a contrastive learning mechanism is applied. This module uses the positive subsequence to learn a representation that is close to the target item's representation, while pushing away representations of irrelevant items. This contrastive loss helps to refine the denoising process and simultaneously estimate user preferences from the cleaned sequence. The delayed reward from the positive subsequence is used to guide the policy learning.
Datasets
The authors evaluated RAP on four publicly available real-world datasets:
- Beauty (Amazon dataset)
- Games (Amazon dataset)
- LastFM (Music listening dataset)
- MovieLens (ML-1m subset)
These datasets span different domains and vary in size and interaction density. The paper follows a standard leave-one-out evaluation strategy, splitting data into training, validation, and testing sets.
Key Insights and Ideas
- Sequence Denoising is Crucial: The paper strongly argues that explicitly filtering out irrelevant historical items is a critical step for improving sequential recommendation performance, going beyond simply down-weighting them.
- Sequential Patterns as Prior Knowledge: Leveraging mined sequential patterns as a form of "collective wisdom" provides valuable guidance to the policy network, enabling it to denoise sequences even without direct supervision signals.
- MDP Formalization for Denoising: Framing denoising as an MDP allows for a principled approach to decision-making at each step of the sequence.
- Contrastive Learning for Robustness: The contrastive learning component helps to make the denoising process more robust by learning from the relationships between relevant and irrelevant items.
Key Results
RAP consistently outperformed several state-of-the-art baselines across all four datasets and two evaluation metrics (HR@10 and NDCG@10). For instance, on the Beauty dataset, RAP achieved an HR@10 of 37.43% and NDCG@10 of 23.75%, significantly outperforming strong baselines like SASRec+ (36.01% HR@10, 25.12% NDCG@10) and BERT4Rec (37.63% HR@10, 26.28% NDCG@10). The paper also includes ablation studies showing that both the pattern enhancement and contrastive learning components are essential for RAP's superior performance, with RAP achieving the best results when both are present. Qualitative analysis demonstrated RAP's ability to correctly identify and recommend items relevant to the user's inferred intent, even when other models like BERT4Rec faltered.
Limitations and Future Work
The authors acknowledge that their current approach relies solely on item interactions for denoising. They suggest that incorporating external knowledge, such as from knowledge graphs, could further enhance the denoising and recommendation process. This would provide richer contextual information and potentially lead to more effective learning.
Summary updated