Abstract
Session-based models are widely used in recommendation systems, typically utilizing Recurrent Neural Networks (RNNs) to process user click sequences. These models generate a vector embedding of the session and use the inner product with item embeddings to score potential interests. While effective for large-scale retrieval using indexing methods like KD-Trees, this approach often fails to capture the diverse interests a user may exhibit within a single session. We propose a novel method that represents session embeddings as symmetric matrices, equivalent to a quadratic form on the item vector space. The score is calculated as the value of the item embedding under this quadratic form. Our experiments demonstrate that this matrix embedding method significantly outperforms traditional vector embedding approaches in retrieval tasks.
Introduction
In large-scale e-commerce platforms, recommendation is generally a two-stage process: Match and Rank. The Match stage involves rapidly retrieving a few thousand candidates from a pool of millions. Current session-based models compress a user's click sequence into a single "session vector." However, this compression introduces a fundamental limitation: the inability to represent independent, diverse interests simultaneously.
The Inherent Defect of Vector Embedding
If a user interacts with multiple distinct categories in one session—for example, browsing both dresses and mobile phones—a single session vector attempts to point toward both. Mathematically, this often results in a vector that points to a "middle ground" representing neither interest accurately. This lack of expressive capacity leads to poor retrieval performance because the vector cannot "point" in two directions at once.
Proposed Method: Matrix Embedding
To overcome this, we introduce the Matrix Embedding Method. Instead of outputting a vector, the RNN (specifically a GRU) generates an output that is reshaped into a symmetric matrix $A$.
Quadratic Scoring
The score for a potential next item $y$ is calculated using a quadratic form: $$Score = y^T Ay$$
This structural shift allows the model to capture multiple distinct interests. According to the spectral theorem, the eigenvectors of the symmetric matrix $A$ corresponding to positive eigenvalues represent the different "directions" of user interest. Thus, a single matrix can effectively represent a union of interests (e.g., both dresses and phones) rather than a mere average.
Implementation and Constraints
To maintain directional consistency during training, we implement an "Upper Half Hyperplane Constraint," ensuring item embeddings are forced into a specific region of the vector space. For efficient retrieval in the Match stage, we utilize two primary strategies:
- Flattening: Mapping the matrix $A$ and item vector $y$ into a higher-dimensional space where the quadratic form becomes a standard inner product.
- Decomposition: Performing approximate searches using the top-$k$ eigenvectors of $A$.
Experimental Results
The model was evaluated on the RSC15 (e-commerce) and Last.fm (music) datasets. The results show a substantial leap in performance over the standard GRU4REC baseline:
- RSC15: The matrix method nearly doubled the recall, moving from a Recall@20 of 0.389 to 0.749.
- Last.fm: The method showed a 6x improvement in recall, increasing from 0.027 to 0.164.
Notably, the matrix method outperformed standard models even when those baselines were given significantly more parameters. This suggests that the structure of the representation—moving from a point to a space—is more critical than raw model size.
Conclusion
By representing user sessions as symmetric matrices rather than vectors, we can better capture the multi-faceted nature of user behavior. This method provides a mathematically elegant and computationally viable way to improve the retrieval stage of recommendation systems, offering both higher accuracy and the potential for more explainable recommendations through eigenvector analysis.
Summary updated