Abstract
In recommendation systems, basket completion involves predicting the next item a user is likely to add to their shopping cart. Traditional approaches using Word2Vec rely on static negative sampling, which often fails to provide a sufficiently strong training signal. This paper introduces GAN-Word2Vec, a novel approach that applies Generative Adversarial Networks (GANs) to the task of basket completion. By replacing static negative sampling with a dynamic, adversarial generator that selects challenging negative examples, the proposed method significantly improves the quality of item embeddings and recommendation accuracy. Experimental results on real-world retail datasets demonstrate that GAN-Word2Vec outperforms standard Word2Vec variants in both Precision@1 and Mean Percentile Rank (MPR).
Introduction
The "basket completion" task is central to modern e-commerce recommendation engines. A common solution is to adapt Word2Vec (or Prod2Vec) to learn item embeddings by distinguishing between observed items in a basket and "negative" items sampled from the catalog. However, the standard Negative Sampling (NS) strategy typically draws from a fixed distribution (e.g., unigram frequency), often resulting in "easy" negatives that do not challenge the model. For instance, distinguishing a gallon of milk from a power tool provides little information for refining item representations. This research addresses the challenge of dynamic negative sampling: identifying "hard" negatives that are plausible enough to force the model to learn finer-grained distinctions.
Methodology
The authors propose GAN-Word2Vec, framing the embedding process as a minimax game between two components:
- The Discriminator ($D$): The core Word2Vec model, which aims to maximize the probability of "real" items while minimizing the probability of "fake" items provided by the generator.
- The Generator ($G$): A model that learns to sample "next-item" candidates that are highly plausible given the current context, aiming to "fool" the discriminator.
Technical Innovations
- Adversarial Negative Sampling Loss: The discriminator uses an extension of the standard Negative Sampling objective, but with negatives drawn dynamically from $G$.
- Policy Gradient Training: Since item selection is discrete and non-differentiable, the authors employ reinforcement learning (specifically policy gradients inspired by the MALIGAN architecture) to train the generator.
- Mixed MLE-MALIGAN Loss: To ensure training stability, a "Mixed Loss" combines the adversarial signal with a standard maximum likelihood estimate (MLE), preventing generator collapse and accelerating convergence.
Experimental Setup
The model was evaluated on two public datasets:
- Amazon Baby Registries: Registries across 15 categories (e.g., 'feeding', 'toys'), containing up to 15,000 baskets per category.
- Belgian Retail Supermarket: A larger dataset with 88,163 baskets and a catalog of 16,470 unique items.
Both datasets utilized an 80/20 training/testing split.
Results and Discussion
The performance of GAN-Word2Vec was compared against standard Word2Vec trained with Noise Contrastive Estimation (NCE) and Negative Sampling (NEG).
As shown in the table above, GAN-Word2Vec achieved superior results across both datasets:
- Amazon Dataset: GAN-Word2Vec reached a Precision@1 of 16.30%, outperforming the NEG baseline (15.40%).
- Belgian Retail Dataset: GAN-Word2Vec achieved a Precision@1 of 35.82%, compared to 34.35% for the NEG baseline.
- Mean Percentile Rank (MPR): The adversarial approach showed statistically significant improvements in MPR, indicating that the true missing item was consistently ranked higher.
Key Insights
- The Value of Hard Negatives: Negatives with high inner-product scores (items the model predicts as related but are absent) provide more informative gradients, leading to more robust representations.
- Stability via Pre-training: Pre-training both $G$ and $D$ using standard negative sampling was found to be essential for high predictive quality.
- Generative Performance: Interestingly, the generator itself often provides better performance on the final basket completion task than the discriminator, likely because the task is inherently generative.
Conclusion and Future Work
While GAN-Word2Vec demonstrates the power of adversarial sampling, the authors note that Word2Vec has limited capacity. Future work will explore scaling this framework to more complex architectures like RNNs, LSTMs, or TextCNNs. Additionally, applying this procedure to models designed for sets, such as Determinantal Point Processes (DPPs), could further improve the modeling of basket diversity.
Summary updated