Matrix embedding method in match for session-based recommendation

Qizhi Zhang, Yi Lin, Kangle Wu, Yongliang Li, Anxiang ZengarXiv 2019 · arXiv · OpenReview

Abstract

Session-based models are widely used in recommendation systems, typically utilizing Recurrent Neural Networks (RNNs) to process user click sequences. These models generate a vector embedding of the session and use the inner product with item embeddings to score potential interests. While effective for large-scale retrieval using indexing methods like KD-Trees, this approach often fails to capture the diverse interests a user may exhibit within a single session. We propose a novel method that represents session embeddings as symmetric matrices, equivalent to a quadratic form on the item vector space. The score is calculated as the value of the item embedding under this quadratic form. Our experiments demonstrate that this matrix embedding method significantly outperforms traditional vector embedding approaches in retrieval tasks.

Introduction

In large-scale e-commerce platforms, recommendation is generally a two-stage process: Match and Rank. The Match stage involves rapidly retrieving a few thousand candidates from a pool of millions. Current session-based models compress a user's click sequence into a single "session vector." However, this compression introduces a fundamental limitation: the inability to represent independent, diverse interests simultaneously.

The Inherent Defect of Vector Embedding

If a user interacts with multiple distinct categories in one session—for example, browsing both dresses and mobile phones—a single session vector attempts to point toward both. Mathematically, this often results in a vector that points to a "middle ground" representing neither interest accurately. This lack of expressive capacity leads to poor retrieval performance because the vector cannot "point" in two directions at once.

Figure 2: The geometric limitation of vector embeddings. A single session vector often ends up in a 'middle ground' between distinct user interests like dresses and phones.

Proposed Method: Matrix Embedding

To overcome this, we introduce the Matrix Embedding Method. Instead of outputting a vector, the RNN (specifically a GRU) generates an output that is reshaped into a symmetric matrix $A$.

Quadratic Scoring

The score for a potential next item $y$ is calculated using a quadratic form: $$Score = y^T Ay$$

This structural shift allows the model to capture multiple distinct interests. According to the spectral theorem, the eigenvectors of the symmetric matrix $A$ corresponding to positive eigenvalues represent the different "directions" of user interest. Thus, a single matrix can effectively represent a union of interests (e.g., both dresses and phones) rather than a mere average.

Figure 3: The proposed Matrix Embedding architecture. The GRU output is reshaped into a symmetrical matrix $A$, and the final score is computed via the quadratic form $y^T Ay$.

Implementation and Constraints

To maintain directional consistency during training, we implement an "Upper Half Hyperplane Constraint," ensuring item embeddings are forced into a specific region of the vector space. For efficient retrieval in the Match stage, we utilize two primary strategies:

  1. Flattening: Mapping the matrix $A$ and item vector $y$ into a higher-dimensional space where the quadratic form becomes a standard inner product.
  2. Decomposition: Performing approximate searches using the top-$k$ eigenvectors of $A$.

Experimental Results

The model was evaluated on the RSC15 (e-commerce) and Last.fm (music) datasets. The results show a substantial leap in performance over the standard GRU4REC baseline:

  • RSC15: The matrix method nearly doubled the recall, moving from a Recall@20 of 0.389 to 0.749.
  • Last.fm: The method showed a 6x improvement in recall, increasing from 0.027 to 0.164.

Notably, the matrix method outperformed standard models even when those baselines were given significantly more parameters. This suggests that the structure of the representation—moving from a point to a space—is more critical than raw model size.

Conclusion

By representing user sessions as symmetric matrices rather than vectors, we can better capture the multi-faceted nature of user behavior. This method provides a mathematically elegant and computationally viable way to improve the retrieval stage of recommendation systems, offering both higher accuracy and the potential for more explainable recommendations through eigenvector analysis.

Summary updated

Abstract

Session based model is widely used in recommend system. It use the user click sequence as input of a Recurrent Neural Network (RNN), and get the output of the RNN network as the vector embedding of the session, and use the inner product of the vector embedding of session and the vector embedding of the next item as the score that is the metric of the interest to the next item. This method can be used for the "match" stage for the recommendation system whose item number is very big by using some index method like KD-Tree or Ball-Tree and etc.. But this method repudiate the variousness of the interest of user in a session. We generated the model to modify the vector embedding of session to a symmetric matrix embedding, that is equivalent to a quadratic form on the vector space of items. The score is builded as the value of the vector embedding of next item under the quadratic form. The eigenvectors of the symmetric matrix embedding corresponding to the positive eigenvalues are conjectured to represent the interests of user in the session. This method can be used for the "match" stage also. The experiments show that this method is better than the method of vector embedding.
BibTeX
@misc{zhang2019matrix,
  author = {Qizhi Zhang and Yi Lin and Kangle Wu and Yongliang Li and Anxiang Zeng},
  title = {{Matrix embedding method in match for session-based recommendation}},
  year = {2019},
  eprint = {1908.10180},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1908.10180},
}
Figure