Position and Distance: Recommendation beyond Matrix Factorization

Shuai Zhang, Lina Yao, Chaoran Huang, Xiwei Xu, Liming Zhu

Introduction

Matrix Factorization (MF) has been a cornerstone of recommender systems for over a decade. By decomposing a user-item interaction matrix into low-dimensional latent factors, MF predicts preferences through the dot product of user and item vectors. However, this approach has a fundamental mathematical limitation: the dot product does not satisfy the triangle inequality. This means that if User A is similar to User B, and User B is similar to User C, MF cannot guarantee that User A and User C will be positioned closely in the latent space. This lack of geometric consistency limits the model's expressiveness and can lead to sub-optimal recommendations.

Metric Factorization Framework

To address these limitations, this paper proposes Metric Factorization, which shifts the recommendation paradigm from "similarity" (dot product) to "position" (Euclidean distance).

The Pipeline

The process begins by converting interaction data into a distance-based representation. For instance, in a 5-star rating system, a high rating (5) is mapped to a small distance (0), while a low rating (1) is mapped to a large distance (4).

The Metric Factorization pipeline: converting similarity matrices to distance matrices for metric factorization into a latent coordinate space.

The model represents users and items as points in a multi-dimensional coordinate system. The predicted preference $\hat{Y}_{ui}$ is defined as the squared Euclidean distance between user point $P_u$ and item point $Q_i$: $$\hat{Y}_{ui} = \|P_u - Q_i\|^2_2$$

Enhancements and Regularization

The framework incorporates several key features to improve robustness:

  • Biases and Confidence: Global, user, and item biases are included to handle varying rating scales. A confidence mechanism weights extreme ratings (1s and 5s) more heavily, reflecting the psychological certainty users have about items they strongly like or dislike.
  • Geometric Constraints: Instead of standard $L_2$ regularization, the model uses Norm Clipping to keep positions within a specific Euclidean ball.
  • Dimension Dropout: Randomly dropping dimensions during distance calculation prevents the model from over-relying on specific latent factors, mitigating overfitting even with high dimensionality.

Geometric Insights

By adhering to the triangle inequality, Metric Factorization naturally captures the neighborhood structure of the data. Similar users are clustered together, and items are positioned relative to the users who prefer them.

Visualization of the latent space during training, showing how items are correctly positioned within user preference boundaries to avoid "invading items."

As shown in the visualization, training helps resolve "invading items"—items that might be incorrectly positioned near a user they don't appeal to—and ensures that items liked by multiple users (e.g., $u_2$ and $u_3$) are placed in overlapping regions of the latent space.

Evaluation and Results

The model was rigorously tested across various datasets, including MovieLens, Goodbooks, YahooMovie, and YahooMusic, covering both rating prediction and item ranking tasks.

  • Rating Prediction: Metric Factorization consistently outperformed traditional BiasedSVD and advanced neural models like NNMF in terms of RMSE and MAE.
  • Item Ranking: On the YahooMovie dataset, the model achieved a 20.8% improvement over Weighted Regularized Matrix Factorization (WRMF) and an 8.7% improvement over Collaborative Metric Learning (CML).
  • Scalability: Performance remains stable as latent dimensions increase, and training efficiency is comparable to standard MF methods.

Conclusion

Metric Factorization provides a mathematically sound and highly expressive alternative to traditional Matrix Factorization. By leveraging Euclidean geometry and pointwise distance prediction, it achieves state-of-the-art results while maintaining computational efficiency. Future work may explore the integration of temporal dynamics and side information to further enhance recommendation quality.

Summary updated

Abstract

For the last two decades, matrix factorization has become one of the fundamental methods for tackling recommendation problems. Recent studies demonstrate that the interaction function dot product used in MF can limit its expressiveness and lead to sub-optimal solutions. In this work, we propose modelling user-item interaction patterns in an alternative way by considering positions and distances of users and items. We assume that users and items can be positioned in a low dimensional space and their explicit closeness can be measured using Euclidean distance metric. In addition, we adopted a weighted strategy to adaptively assign different confidence levels for positive and negative samples, which introduces more flexibility for recommendation with implicit interactions. Comprehensive experiments on multiple real-world datasets demonstrate superior performances of our model over state-of-the-art competing methods including conventional matrix factorization based approaches and recent metric learning based approaches.
BibTeX
@misc{zhang2018position,
  author = {Shuai Zhang and Lina Yao and Chaoran Huang and Xiwei Xu and Liming Zhu},
  title = {{Position and Distance: Recommendation beyond Matrix Factorization}},
  year = {2018},
  url = {https://openreview.net/forum?id=T4oSS3Wr7Zq},
}
Figure