Abstract
Sequential Recommendation (SR) models often struggle with data sparsity and inherent noise in user interaction histories. While Contrastive Learning (CL) has been introduced to mitigate these issues, most existing methods rely on hand-crafted, stochastic data augmentations that are non-adaptive and often produce sub-optimal training signals. This paper presents MCLRec, a novel framework that replaces heuristic augmentations with learnable model augmenters optimized via meta-learning. By adaptively discovering informative "views" of user behavior, MCLRec significantly improves recommendation accuracy, robustness, and training efficiency.
Introduction
Sequential Recommendation (SR) aims to predict a user's next interaction by modeling the temporal patterns in their historical behavior. Despite the success of deep learning architectures like Transformers in this domain, two persistent challenges remain: data sparsity (users with limited interaction history) and noisy data (accidental clicks or irrelevant interactions).
Contrastive Learning (CL) has recently emerged as a promising solution. By generating multiple augmented versions of a single sequence—through techniques like random masking, cropping, or reordering—and forcing the model to recognize these as the same underlying user intent, CL helps the model learn more robust representations. However, these traditional CL methods are often "blind." They rely on fixed, random operations that do not adapt to the specific characteristics of the dataset or the current state of the model. Consequently, they often generate "easy" contrastive pairs that fail to provide a meaningful training signal, or they require massive batch sizes to find sufficiently challenging negative samples.
The MCLRec Framework
MCLRec (Meta-optimized Contrastive Learning for Recommendation) addresses these limitations by introducing a learnable and adaptive augmentation strategy. The framework is built upon three core pillars:
1. Learnable Model Augmentation
Instead of applying stochastic perturbations to the input sequence, MCLRec utilizes two trainable Multi-Layer Perceptrons (MLPs) as model-level augmenters. These augmenters take the intermediate representations from the encoder and transform them into new, learnable views. This shift from discrete data-level augmentation to continuous model-level augmentation allows for a more nuanced and differentiable exploration of the representation space.
2. Meta-Learning Training Strategy
To ensure that the learnable augmenters generate views that actually benefit the recommendation task, MCLRec employs a two-stage meta-optimization process:
- Stage 1 (Encoder Update): The sequence encoder is updated using a joint objective function that combines the primary recommendation loss (e.g., Cross-Entropy) and the contrastive loss.
- Stage 2 (Meta-Update): The augmenters are updated based on the subsequent performance of the encoder. This "learning to learn" approach forces the augmenters to generate views that are specifically difficult and informative, essentially acting as an automated curriculum that challenges the encoder to improve its discriminative power.
3. Contrastive Regularization
A common risk with learnable augmenters is "feature collapse," where the augmenters learn to output identical or trivial embeddings to minimize the contrastive loss without learning useful features. To prevent this, MCLRec introduces a contrastive regularization term. This term enforces a predefined margin between the similarities of positive pairs and negative pairs, ensuring that the generated views remain distinct and discriminative throughout the training process.
Experimental Evaluation
Datasets and Setup
The authors evaluated MCLRec on three widely recognized benchmarks:
- Amazon Sports: 35,598 users, 18,357 items.
- Amazon Beauty: 22,363 users, 12,101 items.
- Yelp: 30,431 users, 20,033 items.
Standard pre-processing was applied, filtering out users and items with fewer than five interactions to maintain data quality.
Performance and Efficiency
MCLRec consistently outperformed state-of-the-art baselines, including SASRec, CL4SRec, and DuoRec. Key findings include:
- Accuracy: The model achieved relative improvements of 3.94% to 8.41% in Hit Ratio (HR@20) and 2.47% to 5.69% in NDCG@20 over the strongest baselines.
- Robustness: When subjected to 15% synthetic noise (randomly inserted negative items), MCLRec exhibited significantly higher stability than standard models like SASRec, maintaining its predictive accuracy despite the corrupted input sequences.
- Computational Efficiency: One of the most striking results was MCLRec's efficiency. While traditional CL methods often require batch sizes of 2048 or higher to be effective, MCLRec with a batch size of 64 outperformed CL4SRec using a batch size of 256. This suggests that meta-optimized views are far more informative than those generated by random sampling.
Conclusion and Future Work
MCLRec demonstrates that moving away from hand-crafted augmentations toward learnable, meta-optimized views can significantly enhance the performance and robustness of sequential recommenders. By treating augmentation as a learnable component of the model architecture, MCLRec effectively navigates the trade-off between data sparsity and noise.
Future research directions include exploring more sophisticated architectures for the augmenters, such as Self-Attention mechanisms, to capture more complex dependencies. Additionally, while the meta-learning approach is efficient in terms of batch size, the computational overhead of calculating second-order derivatives remains an area for optimization.
Summary updated