A Unified Collaborative Representation Learning for Neural-Network Based Recommender Systems

Yuanbo Xu, En Wang, Yongjian Yang, Yi Chang

TL;DR

Most neural recommender systems rely on "inner product" embeddings that fail to capture complex user-item relationships and violate basic geometric principles. This paper introduces Magnetic Metric Learning (MML), a unified framework that uses dual triplets and adaptive margins to pull similar entities together and push dissimilar ones apart in a mathematically consistent latent space, resulting in a 17% average performance boost over state-of-the-art models.

What Problem Does This Paper Address?

Modern recommender systems typically follow a two-stage process: first, they perform Matrix Embedding (ME) to turn users and items into latent vectors, and then they feed these vectors into a neural network to predict preferences.

The authors identify two major flaws in current ME approaches:

  1. The Ideological Disadvantage: Most models only look at direct user-item interactions (e.g., User A bought Item B). They ignore "latent" relationships, such as the fact that two users who give identical ratings to the same set of items should probably have very similar representations.
  2. The Methodological Disadvantage: The industry standard is to use the inner product to measure similarity. However, inner products do not satisfy the triangle inequality (the principle that the direct path between two points is the shortest). This mathematical inconsistency can lead to "chaos" in the latent space, where the relative distances between users and items become unreliable and biased.

Methods

The authors propose Magnetic Metric Learning (MML) to map users and items into a single, unified latent space. Key innovations include:

  • Dual Triplets: Instead of just comparing a user to a "positive" and "negative" item, MML uses dual triplets to model three types of relationships simultaneously: user-to-item, user-to-user, and item-to-item. This "magnetic" approach pulls similar entities together and pushes dissimilar ones away.
  • Weighted Metric (Mahalanobis Distance): Rather than using simple Euclidean distance or inner products, MML learns a weighted metric matrix ($W$). This allows the model to learn which dimensions of the latent vector are most important for different types of relationships.
  • Adaptive Margins: Not all users provide the same amount of data. MML employs adaptive "safety margins" in its loss function. For users or items with very few interactions, the model applies a larger margin to prevent overfitting and ensure robust learning.
  • Covariance Regularization: To prevent the latent vectors from becoming redundant or biased, the authors incorporate a regularization term that encourages the dimensions of the embeddings to be decorrelated.

Datasets

The model was evaluated on four diverse real-world datasets:

  • Amazon (Sports and Outdoors): A standard benchmark for product recommendations.
  • Yelp: Focused on local business reviews and user preferences.
  • Taobao & Jingdong: Large-scale e-commerce datasets from China, used to validate the model's performance on high-sparsity data.
  • Amazon Beauty: A "million-scale" dataset (6.4M users, 1.6M items) used specifically to test the scalability and efficiency of MML.

Key Insights and Ideas

  • Geometry Matters: The "aha moment" here is that the inner product—while computationally cheap—is a poor proxy for similarity in high-dimensional spaces because it lacks the metric properties (like the triangle inequality) needed for stable clustering.
  • Unified Entity Treatment: By treating users and items as the same type of entity in a unified space, MML simplifies the recommendation problem into a pure distance-measuring task. This makes the resulting embeddings much more interpretable.
  • The "Magnetic" Pull of Latent Relationships: The authors show that explicitly forcing similar users to be near each other (even if they haven't interacted with the exact same items yet) significantly relieves the "data sparsity" problem that plagues most recommenders.

Key Results

  • Accuracy Boost: MML outperformed five representative metric learning baselines (like CML and SML) and nine popular recommendation models (like NCF and GraphRec). On average, it achieved a 17% improvement in Hitting Ratio (HR) and Recall.
  • Clustering Purity: Using Normalized Mutual Information (NMI), the authors proved that MML creates much "cleaner" clusters of similar items compared to traditional Matrix Factorization.
  • Scalability: Despite the added complexity of dual triplets, MML proved efficient. On the million-scale Amazon Beauty dataset, it achieved the best performance with a training time comparable to much simpler models.
  • Neural Network Synergy: When used as a pre-processing step for neural recommenders like NCF, MML-trained embeddings provided a significant "head start," leading to higher final accuracy than embeddings learned through standard methods.

Limitations and Future Work

The authors acknowledge a few areas for growth:

  • Threshold Sensitivity: The construction of latent relationship sets depends on a control threshold ($\theta$) for similarity. Finding the optimal $\theta$ currently requires manual tuning.
  • Side Information: While MML is powerful, this paper focuses primarily on the rating matrix. The authors suggest that incorporating "side information"—such as item images, descriptions, or user demographics—into the MML framework is the logical next step to further improve recommendation quality in extremely cold-start scenarios.

Summary updated

Abstract

Most NN-RSs focus on accuracy by building representations from the direct user-item interactions (e.g., user-item rating matrix), while ignoring the underlying relatedness between users and items (e.g., users who rate the same ratings for the same items should be embedded into similar representations), which is an ideological disadvantage. On the other hand, ME models directly employ inner products as a default loss function metric that cannot project users and items into a proper latent space, which is a methodological disadvantage. In this paper, we propose a supervised collaborative representation learning model - Magnetic Metric Learning (MML) - to map users and items into a unified latent vector space, enhancing the representation learning for NN-RSs. Firstly, MML utilizes dual triplets to model not only the observed relationships between users and items, but also the underlying relationships between users as well as items to overcome the ideological disadvantage. Specifically, a modified metric-based dual loss function is proposed in MML to gather similar entities and disperse the dissimilar ones. With MML, we can easily compare all the relationships (user to user, item to item, user to item) according to the weighted metric, which overcomes the methodological disadvantage. We conduct extensive experiments on four real-world datasets with large item space. The results demonstrate that MML can learn a proper unified latent space for representations from the user-item matrix with high accuracy and effectiveness, and lead to a performance gain over the state-of-the-art RS models by an average of 17%.
BibTeX
@article{xu2022unified,
  author = {Yuanbo Xu and En Wang and Yongjian Yang and Yi Chang},
  title = {{A Unified Collaborative Representation Learning for Neural-Network Based Recommender Systems}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2022},
  doi = {10.1109/tkde.2021.3054782},
}

Loading related content...

Figure