Abstract
This paper introduces MF-VMLP, a hybrid recommendation framework that integrates traditional Matrix Factorization (MF) with a Visual Multi-Layer Perceptron (VMLP). By leveraging high-dimensional visual features extracted from item images via pre-trained Convolutional Neural Networks (CNNs), the model captures complex, non-linear user preferences for visually-driven items. Experimental results on Amazon datasets demonstrate significant improvements in rating prediction accuracy for fashion categories—up to 7.3%—while highlighting the domain-specific utility of visual information, as evidenced by zero improvement in functional categories like electronics.
Introduction
Traditional recommender systems primarily rely on collaborative filtering, utilizing explicit or implicit feedback to predict user preferences. However, these methods often overlook a critical driver of consumer behavior: the visual appearance of products. While recent research has attempted to incorporate visual data, existing models frequently rely on simple linear transformations or struggle to effectively combine explicit rating data with deep, non-linear architectures. This paper addresses these limitations by proposing a hybrid neural network framework that seamlessly integrates high-dimensional visual features with traditional collaborative filtering.
Methodology
The proposed MF-VMLP model builds upon the Neural Collaborative Filtering (NCF) framework, employing a dual-pathway architecture to capture both linear and non-linear interactions.
Visual Feature Extraction
The system utilizes a pre-trained Convolutional Neural Network (CNN) to extract a 4096-dimensional visual feature vector for each item. To manage this high dimensionality, an embedding kernel projects these features into a lower-dimensional latent space.
Hybrid Architecture
The model fuses two distinct components:
- Visual Matrix Factorization (VMF): An extension of standard MF that includes a linear visual interaction term.
- Visual Multi-Layer Perceptron (VMLP): A deep neural network that processes concatenated user latent vectors, item latent vectors, and reduced visual features through multiple layers with ReLU activations to capture complex interactions.
A key innovation is the use of decoupled embeddings. Rather than forcing both pathways to share the same embedding space, MF-VMLP maintains separate embeddings for MF and VMLP. This allows each component to optimize its representation independently before their final hidden layers are concatenated and passed through a fully connected layer to predict the rating.
Results and Discussion
The model was evaluated on three Amazon datasets: Women's Clothing, Men's Clothing, and Cell Phones. Performance was measured using Root Mean Square Error (RMSE).
The results demonstrate a clear domain dependency for visual features. In the Amazon Women category, MF-VMLP achieved an RMSE of 1.0575, representing a 7.3% improvement over standard MF. Similarly, the Amazon Men category saw a 5.2% improvement. Conversely, the model showed 0% improvement for the Amazon Phones dataset. This confirms that the model effectively learns visual aesthetics where they are relevant (fashion) rather than exploiting statistical noise.
Conclusion
The MF-VMLP framework successfully demonstrates that integrating visual information through non-linear neural pathways significantly enhances recommendation accuracy in visually-driven domains. The use of decoupled embeddings and deep MLP layers proves essential for capturing the subjective nature of visual preference. Future work will explore domain-specific visual extractors and the integration of multi-modal data, such as textual reviews and social context, to further refine recommendation performance.
Summary updated