Sequential Modeling with Multiple Attributes for Watchlist Recommendation in E-Commerce

Uriel Singer, Haggai Roitman, Yotam Eshel, Alexander Nus, Ido Guy, Or Levi, Idan Hasson, Eliyahu Kiperwasser

TL;DR

This paper introduces Watchlist Recommendation (WLR), a specialized sequential recommendation task focused on prioritizing items a user has already saved. To solve it, the authors propose Trans2D, a Transformer-based model featuring a novel Attention2D mechanism that learns complex relationships across both the sequence of items and their individual attributes (like price and category) simultaneously.

What Problem Does This Paper Address?

In e-commerce, "watchlists" are ubiquitous tools for tracking items. However, active users often accumulate dozens of items, making it overwhelming to track price drops, stock changes, or shifting personal priorities.

Standard Sequential Recommendation (SR) models are designed to predict the identity of the next item a user will interact with from a massive, static catalog. The authors argue that Watchlist Recommendation is different:

  1. Dynamic Recall Set: The "candidate" items are limited to what is currently in the user's watchlist, which changes as they add or remove items.
  2. Attribute Sensitivity: Watchlist items are often tracked specifically because their attributes (like price) are dynamic.
  3. Sparsity: Representing items solely by IDs leads to overfitting because many items appear only a few times.

The gap lies in the fact that most existing models either ignore item attributes or "collapse" them (via pooling or concatenation) before the sequence modeling step, losing the fine-grained interactions between specific attributes over time.

Methods

The authors propose Trans2D, which treats user history not as a 1D sequence of item IDs, but as a 2D array where one axis is the sequence (time) and the other is the channel (attributes).

Key Innovations:

  • Attention2D: Instead of standard self-attention, this layer calculates three types of scores:
  • $A^F$ (Point-to-Point): Attention between a specific attribute of one item and a specific attribute of another.
  • $A^I$ (Item-to-Item): Marginal attention between whole items, regardless of specific attributes.
  • $A^C$ (Channel-to-Channel): Marginal attention between attribute types (e.g., how much does "price" generally influence "category" preferences?).
  • Linear2D Layers: These layers apply weights per attribute channel, allowing the model to preserve the unique semantics of different attributes (like price vs. condition) throughout the network.
  • Hybrid History: The model incorporates both "watchlist clicks" (explicit interest) and "recently viewed items" (RVIs, providing implicit context) to capture shifts in user intent.

Datasets

The model was evaluated using a large-scale dataset from eBay:

  • Scope: Two weeks of data from February 2021.
  • Scale: 40,344 active users and over 5.3 million items.
  • Attributes: 16 attributes per item, including item-ID, price (binned), seller-ID, condition, category levels, and site-ID.
  • Benchmarks: The authors compared Trans2D against a wide array of adapted baselines, including GRU4Rec, SASRec, BST, and FDSA.

Key Insights and Ideas

  • Attributes > IDs: In sparse e-commerce environments, representing items by their attributes allows the model to generalize better to "cold-start" items that weren't seen much during training.
  • Avoid Early Pooling: Most models aggregate attributes into a single vector before the Transformer layers. Trans2D shows that keeping attributes separate until the very end (the prediction layer) allows the attention mechanism to discover much richer patterns (e.g., a price drop in one category influencing interest in a different but related category).
  • Short-Term Signals Rule: The ablation study showed that sequence length and "days considered" have a point of diminishing returns. Most of the "signal" for what a user will click next comes from their very recent interactions.
  • Position Matters: The "Relative Snapshot Position" (how recently an item was added to the watchlist) is one of the most powerful predictors of future clicks.

Key Results

Trans2D significantly outperformed all baselines across all metrics:

  • Precision@1: Trans2D achieved 43.51%, compared to the next best baseline (GRU with concatenated attributes) at 39.08%.
  • NDCG@5: Trans2D reached 26.12, while the best Transformer-based baseline (BST) trailed at 24.94.
  • Qualitative Success: Visualization of the attention weights showed the model successfully "attending" to items in the same leaf-category or items with similar price points when making a prediction.

Limitations and Future Work

  • Feedback Types: The current study focuses on clicks and views. The authors suggest that incorporating search queries and actual purchase history could further refine the "intent" signal.
  • Generalization: While tested on e-commerce, the Attention2D mechanism is theoretically applicable to any sequential task with multi-attribute inputs (like stock market prediction or news recommendation), but this hasn't been tested yet.
  • Computational Cost: While the authors claim only a "slight expense of parameters," 2D attention is inherently more complex than 1D, which might impact latency in extremely high-throughput real-time systems.

Summary updated

Abstract

In e-commerce, the watchlist enables users to track items over time and has emerged as a primary feature, playing an important role in users' shopping journey. Watchlist items typically have multiple attributes whose values may change over time (e.g., price, quantity). Since many users accumulate dozens of items on their watchlist, and since shopping intents change over time, recommending the top watchlist items in a given context can be valuable. In this work, we study the watchlist functionality in e-commerce and introduce a novel watchlist recommendation task. Our goal is to prioritize which watchlist items the user should pay attention to next by predicting the next items the user will click. We cast this task as a specialized sequential recommendation task and discuss its characteristics. Our proposed recommendation model, Trans2D, is built on top of the Transformer architecture, where we further suggest a novel extended attention mechanism (Attention2D) that allows to learn complex item-item, attribute-attribute and item-attribute patterns from sequential-data with multiple item attributes. Using a large-scale watchlist dataset from eBay, we evaluate our proposed model, where we demonstrate its superiority compared to multiple state-of-the-art baselines, many of which are adapted for this task.
BibTeX
@inproceedings{singer2022sequential,
  author = {Uriel Singer and Haggai Roitman and Yotam Eshel and Alexander Nus and Ido Guy and Or Levi and Idan Hasson and Eliyahu Kiperwasser},
  title = {{Sequential Modeling with Multiple Attributes for Watchlist Recommendation in E-Commerce}},
  booktitle = {WSDM},
  year = {2022},
  doi = {10.1145/3488560.3498453},
}
Figure