HAM: Hybrid Associations Model with Pooling for Sequential Recommendation

Bo Peng, Zhiyun Ren, Srinivasan Parthasarathy, Xia NingarXiv 2020 · arXiv

Abstract

Sequential recommendation aims to predict a user's next interaction based on their historical sequence. While modern research favors complex deep learning architectures like Recurrent Neural Networks (RNNs) and Attention mechanisms, these models often struggle with the inherent sparsity of recommendation data. We present the Hybrid Associations Model (HAM), which simplifies the recommendation process by utilizing intuitive pooling operations and explicit item synergies. HAM significantly outperforms state-of-the-art methods, achieving up to a 46.6% improvement in accuracy and a 139.7x speedup in execution time.

Introduction

The current landscape of sequential recommendation is dominated by highly parameterized models. However, recommendation data is typically sparse, with most users having few interactions. In such "data-starved" environments, complex models often learn noise rather than meaningful patterns. HAM addresses this by moving away from "overthinking" architectures toward a more explicit and efficient approach that leverages general user tastes, multi-scale sequential patterns, and item-level synergies.

Methodology

The HAM architecture generates recommendation scores by integrating three primary factors:

  1. User Long-Term Preferences: Standard user embeddings capture broad interests (e.g., genre preferences).
  2. Sequential Association Patterns: Instead of complex gating, HAM employs simplistic mean or max pooling to represent recent history across two scales: "high-order" (longer window) and "low-order" (immediate interactions).
  3. Item Synergies: To capture "union-level effects"—where a combination of items implies a specific intent—HAM uses the Hadamard product (element-wise multiplication) of item embeddings.
The HAM architecture featuring Embedding Lookup, Pooling, and Prediction layers.

The model is optimized using a Bayesian Personalized Ranking (BPR) objective to prioritize true next items over random negative samples.

Experimental Setup

HAM was evaluated on six public benchmark datasets with varying levels of sparsity: Amazon (CDs, Books), Goodreads (Children, Comics), and MovieLens (1M, 20M). Three experimental protocols (80-20 split, 80-3 split, and Leave-3-out) were used to ensure robustness.

Results and Discussion

HAM consistently demonstrates superior performance across diverse datasets. As shown in the Recall metrics, HAM often provides the best or second-best results, particularly on sparse datasets like Amazon CDs, where it showed a 12.8% improvement over the strongest baseline.

Recall@5 and Recall@10 performance of HAM compared to state-of-the-art baselines across six datasets.

Key Insights

  • The "Attention" Illusion: Analysis reveals that in sparse settings, attention weights often converge to uniform values, effectively performing simple averaging. HAM's use of mean pooling achieves similar or better results without the computational overhead.
  • Synergy as a Latent Cross: Explicitly modeling item interactions via the Hadamard product allows the model to amplify shared features, capturing user intent more effectively than additive methods.
  • Efficiency and Robustness: HAM achieved a massive 139.7x speedup over Caser and remained stable across hyperparameter settings, unlike SASRec, which showed high sensitivity.

Conclusion

HAM challenges the "deeper is better" paradigm in recommender systems. By prioritizing simplicity and explicit association modeling, it achieves state-of-the-art accuracy with significantly lower computational costs. Future work will explore the interaction between different pooling strategies and item synergies to further refine the model's performance on dense datasets.

Summary updated

Abstract

Sequential recommendation aims to identify and recommend the next few items for a user that the user is most likely to purchase/review, given the user's purchase/rating trajectories. It becomes an effective tool to help users select favorite items from a variety of options. In this manuscript, we developed hybrid associations models (HAM) to generate sequential recommendations using three factors: 1) users' long-term preferences, 2) sequential, high-order and low-order association patterns in the users' most recent purchases/ratings, and 3) synergies among those items. HAM uses simplistic pooling to represent a set of items in the associations, and element-wise product to represent item synergies of arbitrary orders. We compared HAM models with the most recent, state-of-the-art methods on six public benchmark datasets in three different experimental settings. Our experimental results demonstrate that HAM models significantly outperform the state of the art in all the experimental settings, with an improvement as much as 46.6%. In addition, our run-time performance comparison in testing demonstrates that HAM models are much more efficient than the state-of-the-art methods, and are able to achieve significant speedup as much as 139.7 folds.
BibTeX
@misc{peng2020ham,
  author = {Bo Peng and Zhiyun Ren and Srinivasan Parthasarathy and Xia Ning},
  title = {{HAM: Hybrid Associations Model with Pooling for Sequential Recommendation}},
  year = {2020},
  eprint = {2002.11890},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2002.11890},
}
Figure