Abstract
Sequential recommendation (SR) has traditionally relied on generative modeling, specifically Next-Item Prediction (NIP), to capture user preferences. However, these methods often struggle with data sparsity, high computational costs associated with large item catalogs, and ineffective data augmentation. This paper introduces ELECRec, a framework that reimagines sequential recommendation as a discriminative task. By training a discriminator to distinguish between actual user interactions and "plausible fakes" generated by an auxiliary model, ELECRec learns more robust item representations. Experimental results across multiple benchmarks demonstrate that ELECRec significantly outperforms state-of-the-art generative models in both accuracy and training efficiency.
Introduction
Sequential recommendation systems aim to predict a user's future interests based on their historical interactions. The dominant paradigm for these systems is generative: given a sequence of items, the model is trained to predict the exact next item in the series. While effective, this approach faces significant hurdles. First, the "Next-Item Prediction" (NIP) task is highly sensitive to data sparsity; if a specific transition is rare, the model fails to learn the underlying correlation. Second, calculating a Softmax probability over millions of items is computationally prohibitive. Finally, standard data augmentation techniques often introduce noise that degrades model performance.
ELECRec addresses these issues by shifting the objective from generation to discrimination. Inspired by the ELECTRA architecture in natural language processing, ELECRec trains a recommender to identify whether an item in a sequence is an original user interaction or a replacement suggested by a generator.
The ELECRec Framework
The ELECRec architecture consists of three primary components that work in tandem to refine the recommendation process:
1. The Generator ($G$)
The generator is a standard Transformer-based recommender trained on the traditional NIP task. Its role is not to provide the final recommendations but to serve as a source of "plausible" alternative items. By suggesting items that are contextually relevant but not the actual next item, the generator creates a challenging environment for the discriminator.
2. The Data Sampler
This component modifies the original user sequences. It takes a real sequence and replaces a specific percentage (defined by the hyperparameter $\alpha$) of the items with suggestions produced by the generator. This creates a corrupted sequence where some items are "real" and others are "fakes."
3. The Discriminator ($D$)
The discriminator is the core of the ELECRec system. It receives the corrupted sequence and performs binary classification for every position in the sequence, answering the question: "Is this the original item, or a replacement?"
Unlike Generative Adversarial Networks (GANs), which are often unstable due to the adversarial nature of the generator and discriminator, ELECRec uses a generator trained via maximum likelihood. This ensures the generator provides a "steadily lifting" challenge, leading to stable training and superior representation learning.
Experimental Setup
The authors evaluated ELECRec against several state-of-the-art baselines, including SASRec, BERT4Rec, and $S^3$-Rec. The evaluation spanned four diverse datasets:
- Yelp: Business and service interactions.
- Amazon Sports: Sporting goods purchases.
- Amazon Beauty: Cosmetic and beauty product interactions.
- Amazon Toys: Toy and game purchases.
To ensure data quality, all datasets were pre-processed to include only users and items with at least five interactions, mitigating the impact of extreme cold-start noise.
Key Insights and Results
The Power of Plausible Negatives
Traditional discriminative models often rely on random negative sampling, which provides a weak training signal because random items are too easy to distinguish from the truth. By using a "warm" generator, ELECRec forces the discriminator to learn fine-grained nuances and subtle correlations between items, leading to more precise latent representations.
Efficiency and Scalability
By transforming the problem into a binary classification task, ELECRec avoids the heavy computational overhead of performing a full Softmax over the entire item vocabulary. This makes the model significantly more scalable for large-scale industrial applications.
Performance Metrics
ELECRec consistently outperformed all baselines across Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG). Notable highlights include:
- Yelp Dataset: ELECRec achieved a staggering 137.16% improvement in HR@5 and a 66.67% increase in NDCG@10 compared to the strongest baseline.
- Convergence: The model reached peak performance levels in fewer training epochs and less wall-clock time than SASRec, demonstrating superior training efficiency.
- Sampling Strategy: Sensitivity analysis revealed that replacing approximately 50% ($\alpha = 0.5$) of the items in a sequence typically yields the best results, providing the optimal balance of difficulty for the discriminator.
Conclusion and Future Work
ELECRec demonstrates that "recognizing" the correct item is often a more effective training signal for sequential recommendation than "generating" it. By leveraging a discriminative objective and shared item embeddings, the framework overcomes the limitations of data sparsity and computational cost inherent in NIP-based models.
While highly effective, the model's performance is sensitive to hyperparameters like the sampling rate ($\alpha$) and the weight of the discriminator loss ($\lambda$). Future research may explore automated hyperparameter tuning and the application of discriminative training to other recommendation domains beyond sequential behavior, such as graph-based or multi-modal recommendation systems.
Summary updated