A Unified Collaborative Representation Learning for Neural-Network based Recommender Systems

Yuanbo Xu, En Wang, Yongjian Yang, Yi Chang

Abstract

Most Neural Network-based Recommender Systems (NN-RSs) focus on accuracy by building representations from direct user-item interactions, often ignoring the underlying relatedness between entities. Furthermore, traditional Matrix Embedding (ME) models rely on inner products, which fail to project users and items into a mathematically consistent latent space. We propose Magnetic Metric Learning (MML), a supervised collaborative representation learning model that maps users and items into a unified latent vector space. MML utilizes dual triplets to model both observed user-item interactions and latent relationships between users and items. Extensive experiments on four real-world datasets demonstrate that MML achieves a performance gain over state-of-the-art models by an average of 17%.

Introduction

Modern recommender systems typically follow a two-stage process: first, they perform Matrix Embedding (ME) to transform users and items into latent vectors, and then they feed these vectors into a neural network to predict preferences. However, current ME approaches suffer from two major flaws:

  1. The Ideological Disadvantage: Most models only consider direct user-item interactions (e.g., User A bought Item B). They ignore "latent" relationships, such as the fact that two users who provide identical ratings for the same items should have similar representations.
  2. The Methodological Disadvantage: The industry standard is to use the inner product to measure similarity. However, inner products do not satisfy the triangle inequality, leading to a "chaotic" latent space where relative distances are unreliable and biased.

Magnetic Metric Learning (MML)

To address these issues, we propose Magnetic Metric Learning (MML). MML simplifies the recommendation problem into a pure distance-measuring task by treating users and items as the same type of entity in a unified space. Key innovations include:

  • Dual Triplets: Instead of just comparing a user to a "positive" and "negative" item, MML uses dual triplets to model user-to-item, user-to-user, and item-to-item relationships simultaneously. This "magnetic" approach pulls similar entities together and pushes dissimilar ones away.
  • Weighted Metric (Mahalanobis Distance): Rather than using simple Euclidean distance, MML learns a weighted metric matrix ($W$), allowing the model to identify which dimensions of the latent vector are most important for different relationship types.
  • Adaptive Margins: MML employs adaptive "safety margins" in its loss function. For users or items with sparse data, the model applies a larger margin to prevent overfitting and ensure robust learning.
  • Covariance Regularization: To prevent redundant or biased latent vectors, a regularization term encourages the dimensions of the embeddings to be decorrelated.
Comparison of CML and MML frameworks, highlighting the "pull" and "push" effects on latent relationships.

Experimental Results

We evaluated MML on four diverse real-world datasets: Amazon (Sports and Outdoors), Yelp, Taobao, and Jingdong. MML was compared against five metric learning baselines (e.g., CML, SML) and nine popular recommendation models (e.g., NCF, GraphRec).

Performance comparison across four datasets showing MML's superiority in Hitting Ratio/Recall as latent dimensions increase.

On average, MML achieved a 17% improvement in Hitting Ratio (HR) and Recall. Key findings include:

  • Clustering Purity: Using Normalized Mutual Information (NMI), we proved that MML creates much "cleaner" clusters of similar items compared to traditional Matrix Factorization.
  • Scalability: Despite the added complexity of dual triplets, MML proved efficient on the million-scale Amazon Beauty dataset, achieving top performance with training times comparable to simpler models.
  • Neural Network Synergy: When used as a pre-processing step for neural recommenders like NCF, MML-trained embeddings provided a significant "head start," leading to higher final accuracy than standard methods.

Conclusion

MML demonstrates that the geometry of the latent space is critical for recommendation performance. By replacing the inner product with a metric-based approach and explicitly modeling latent relationships, MML significantly mitigates the data sparsity problem. Future work will explore automated threshold tuning for similarity sets and the integration of side information, such as item images and user demographics, to further enhance performance in cold-start scenarios.

Summary updated

Abstract

Most NN-RSs focus on accuracy by building representations from the direct user-item interactions (e.g., user-item rating matrix), while ignoring the underlying relatedness between users and items (e.g., users who rate the same ratings for the same items should be embedded into similar representations), which is an ideological disadvantage. On the other hand, ME models directly employ inner products as a default loss function metric that cannot project users and items into a proper latent space, which is a methodological disadvantage. In this paper, we propose a supervised collaborative representation learning model - Magnetic Metric Learning (MML) - to map users and items into a unified latent vector space, enhancing the representation learning for NN-RSs. Firstly, MML utilizes dual triplets to model not only the observed relationships between users and items, but also the underlying relationships between users as well as items to overcome the ideological disadvantage. Specifically, a modified metric-based dual loss function is proposed in MML to gather similar entities and disperse the dissimilar ones. With MML, we can easily compare all the relationships (user to user, item to item, user to item) according to the weighted metric, which overcomes the methodological disadvantage. We conduct extensive experiments on four real-world datasets with large item space. The results demonstrate that MML can learn a proper unified latent space for representations from the user-item matrix with high accuracy and effectiveness, and lead to a performance gain over the state-of-the-art RS models by an average of 17%.
BibTeX
@article{xu2022unified,
  author = {Yuanbo Xu and En Wang and Yongjian Yang and Yi Chang},
  title = {{A Unified Collaborative Representation Learning for Neural-Network based Recommender Systems}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2022},
  doi = {10.1109/tkde.2021.3054782},
}
Figure