Autoregressive models for session-based recommendations using set expansion

Teresa Yu, Zhou Xiang-hong, Xiaotie DengPeerJ 2025 · DOI

TL;DR

This paper introduces DSETRec, a novel session-based recommendation system that treats user interaction sequences as unordered sets, overcoming limitations of traditional sequence-aware models. By focusing on item co-occurrence patterns rather than strict order, DSETRec achieves significant performance improvements, particularly in scenarios with noisy or ambiguous interaction sequences.

What Problem Does This Paper Address?

Modern recommendation systems often rely on understanding user behavior within a "session" – a series of interactions within a short timeframe. The goal is to predict what a user might be interested in next. Traditional approaches, including deep learning methods like RNNs and Transformers, excel at capturing the sequential order of these interactions. However, this reliance on order can be a weakness. In many real-world scenarios, such as e-commerce browsing or video streaming, user interactions aren't always strictly sequential. Users might jump between items, revisit previous ones, or their actions might be influenced by broader co-occurrence patterns rather than a strict step-by-step progression. When interaction order is ambiguous or unreliable, sequence-based models can struggle, leading to suboptimal recommendations. This paper aims to address this by proposing a model that can effectively recommend items even when the precise order of interactions is not crucial or is difficult to discern.

Methods

The core innovation of this paper is the Deep Set Session-based Recommendation (DSETRec) model, which reframes session-based recommendation as a set learning problem. Instead of processing interactions as a sequence, DSETRec treats them as an unordered set.

Key components of DSETRec include:

  • Set Representation: User interactions within a session are represented as a set, ensuring permutation invariance (the order of items doesn't affect the outcome).
  • Embedding Layer: Item IDs are converted into dense, low-dimensional vectors to capture their semantic meaning.
  • Set Extension (Approximator-Aggregator): This is the heart of the model. It uses a structure inspired by DeepSets (Zaheer et al., 2017) to learn co-occurrence features among items in the set. It involves an "approximator" that processes individual elements and an "aggregator" that combines these processed elements to form a session-level representation. This component focuses on capturing "co-occurrence dependency" between items.
  • Autoregressive Prediction: DSETRec employs a multi-stage autoregressive approach. In each stage, it masks known elements of the session, predicts missing items using the set extension mechanism, and then adds these predicted items to the set for the next stage. This iterative process allows the model to gradually build a recommendation sequence.
  • Prediction Layer: A final linear layer maps the aggregated co-occurrence features to probabilities for all items in the vocabulary, enabling the selection of the most likely next item.

The model's training utilizes a Top 1 loss function, which focuses on correctly ranking the target item.

Datasets

The proposed DSETRec model was evaluated on two publicly available datasets:

  • Yoochoose: This dataset originates from the RecSys 2015 Data Mining Challenge and contains click and purchase events from e-commerce user sessions. The paper uses two versions: Yoochoose (1/64) and Yoochoose (1/4).
  • Diginetica: This dataset was sourced from the 2016 CIKM Cup and primarily includes transaction data.

The paper does not introduce new benchmarks but uses these established datasets to compare its performance against existing methods.

Key Insights and Ideas

  • Order Doesn't Always Matter: The most significant insight is that for many session-based recommendation tasks, the strict sequential order of interactions can be less important than the overall co-occurrence patterns between items. Ignoring order can lead to more robust and accurate recommendations.
  • Set Learning for Recommendations: Framing session-based recommendation as a set learning problem is a novel approach that effectively handles unordered data and captures item relationships beyond simple sequences.
  • Co-occurrence is Key: The model's success hinges on its ability to capture "co-occurrence dependency," which represents how items tend to appear together in a session, regardless of their exact order. This is a more nuanced understanding of user intent than just sequential transitions.
  • Generalizability: The set-based approach demonstrates strong adaptability to sessions of varying lengths (short, medium, and long), suggesting its broad applicability.

Key Results

  • Outperforms Sequence-Based Variants: DSETRec achieved significant improvements over its sequence-based counterpart on the Yoochoose dataset, with a 13.2% increase in P@20 and an 11.85% increase in MRR@20.
  • Superior to Baselines: DSETRec consistently outperformed most traditional and advanced sequence-based recommendation models (like POP, Item-KNN, GRU4REC, STAMP, and SR-GNN) across both P@20 and MRR@20 metrics on the tested datasets.
  • Robustness: The model showed high stability across different session lengths, with only a minor performance drop on long sessions compared to overall performance.
  • Ablation Study: Removing either the set extension module or the autoregressive module led to a noticeable performance decline, with the set extension module having a more significant impact, highlighting its critical role.

Limitations and Future Work

  • Hyperparameter Sensitivity: While the model performs well, its performance can be sensitive to hyperparameters like dimensionality. The paper notes that higher dimensions generally improve performance up to a point (around 256 dimensions), after which gains diminish or even reverse. Batch size also has a noticeable impact, with smaller sizes often performing better.
  • Aggregation Methods: The choice of aggregation operations (averaging and summation) significantly impacts performance, with a combination of averaging followed by summation yielding the best results.
  • Future Work: The authors suggest extending the set-based paradigm to other recommendation tasks, such as multi-modal recommendations and cross-domain recommendations. They also propose incorporating mechanisms to enhance interpretability and scalability.

Summary updated

Abstract

With the rapid growth of internet technologies, session-based recommendation systems have emerged as a key paradigm in delivering personalized recommendations by capturing users' dynamic and short-term preferences. Traditional methods predominantly rely on modeling the sequential order of user interactions, deep learning approaches like recurrent neural networks and Transformer architectures. However, these sequence-based models often struggle in scenarios where the order of interactions is ambiguous or unreliable, limiting their real-world applicability. To address this challenge, we propose a novel session-based recommendation model, Deep Set Session-based Recommendation (DSETRec), which approaches the problem from a set-based perspective, eliminating dependence on the interaction sequence. By conceptualizing session data as unordered sets, our model captures the coupling relationships and co-occurrence patterns between items, enhancing prediction accuracy in settings where sequential information is either unavailable or noisy. The model is implemented using a deep autoregressive framework that iteratively masks known elements within a session, predicting and reconstructing additional items based on set data characteristics. Extensive experiments on benchmark datasets show that DSETRec achieves outperforms state-of-the-art baselines. DSETRec achieves a 13.2% and 11.85% improvement in P@20 and MRR@20, respectively, over its sequence-based variant on Yoochoose. Additionally, DSETRec generalizes effectively across both further short and long sessions. These results highlight the robustness of the set-based approach in capturing unordered interaction patterns and adapting to diverse session lengths. This finding provides a foundation for developing more flexible and generalized session-based recommendation systems.
BibTeX
@inproceedings{yu2025autoregressive,
  author = {Teresa Yu and Zhou Xiang-hong and Xiaotie Deng},
  title = {{Autoregressive models for session-based recommendations using set expansion}},
  booktitle = {PeerJ},
  year = {2025},
  doi = {10.7717/peerj-cs.2734},
}
Figure