Introduction
Matrix Factorization (MF) has been a cornerstone of recommender systems for over a decade. By decomposing a user-item interaction matrix into low-dimensional latent factors, MF predicts preferences through the dot product of user and item vectors. However, this approach has a fundamental mathematical limitation: the dot product does not satisfy the triangle inequality. This means that if User A is similar to User B, and User B is similar to User C, MF cannot guarantee that User A and User C will be positioned closely in the latent space. This lack of geometric consistency limits the model's expressiveness and can lead to sub-optimal recommendations.
Metric Factorization Framework
To address these limitations, this paper proposes Metric Factorization, which shifts the recommendation paradigm from "similarity" (dot product) to "position" (Euclidean distance).
The Pipeline
The process begins by converting interaction data into a distance-based representation. For instance, in a 5-star rating system, a high rating (5) is mapped to a small distance (0), while a low rating (1) is mapped to a large distance (4).
The model represents users and items as points in a multi-dimensional coordinate system. The predicted preference $\hat{Y}_{ui}$ is defined as the squared Euclidean distance between user point $P_u$ and item point $Q_i$: $$\hat{Y}_{ui} = \|P_u - Q_i\|^2_2$$
Enhancements and Regularization
The framework incorporates several key features to improve robustness:
- Biases and Confidence: Global, user, and item biases are included to handle varying rating scales. A confidence mechanism weights extreme ratings (1s and 5s) more heavily, reflecting the psychological certainty users have about items they strongly like or dislike.
- Geometric Constraints: Instead of standard $L_2$ regularization, the model uses Norm Clipping to keep positions within a specific Euclidean ball.
- Dimension Dropout: Randomly dropping dimensions during distance calculation prevents the model from over-relying on specific latent factors, mitigating overfitting even with high dimensionality.
Geometric Insights
By adhering to the triangle inequality, Metric Factorization naturally captures the neighborhood structure of the data. Similar users are clustered together, and items are positioned relative to the users who prefer them.
As shown in the visualization, training helps resolve "invading items"—items that might be incorrectly positioned near a user they don't appeal to—and ensures that items liked by multiple users (e.g., $u_2$ and $u_3$) are placed in overlapping regions of the latent space.
Evaluation and Results
The model was rigorously tested across various datasets, including MovieLens, Goodbooks, YahooMovie, and YahooMusic, covering both rating prediction and item ranking tasks.
- Rating Prediction: Metric Factorization consistently outperformed traditional BiasedSVD and advanced neural models like NNMF in terms of RMSE and MAE.
- Item Ranking: On the YahooMovie dataset, the model achieved a 20.8% improvement over Weighted Regularized Matrix Factorization (WRMF) and an 8.7% improvement over Collaborative Metric Learning (CML).
- Scalability: Performance remains stable as latent dimensions increase, and training efficiency is comparable to standard MF methods.
Conclusion
Metric Factorization provides a mathematically sound and highly expressive alternative to traditional Matrix Factorization. By leveraging Euclidean geometry and pointwise distance prediction, it achieves state-of-the-art results while maintaining computational efficiency. Future work may explore the integration of temporal dynamics and side information to further enhance recommendation quality.
Summary updated