Abstract
The sparsity of data is one of the main reasons restricting the performance of recommender systems. In order to solve the sparsity problem, some recommender systems use auxiliary information, especially text information, as a supplement to increase the prediction accuracy of the ratings. However, the two mainstream approaches based on text analysis have some limitations. The bag-of-words-based model is one of them, being difficult to use the contextual information of the paragraph effectively so that only the shallow understanding of paragraph can be parsed. Another model based on deep learning can extract the contextual information of the paragraph, but it also increases the complexity of the model. This paper proposes a novel context-aware recommendation model named paragraph vector matrix factorization (P2VMF) which integrates the unsupervised learning of paragraph embeddings into probabilistic matrix factorization (PMF). Therefore, P2VMF can capture the semantic information of the paragraph and can improve the prediction accuracy of the ratings. Our extensive experiments on real-world datasets show that the performance of the P2VMF model is preferable as compared with those multiple recommendation models in the situation, where the ratings are quite sparse. And we also verified that the P2V part of the model can well express the semantics in the form of vectors.
Introduction
Recommender systems face a perennial challenge: data sparsity. Most users only rate a tiny fraction of available items, making it difficult for standard collaborative filtering—which relies on rating patterns—to predict preferences accurately. To solve this, researchers often use auxiliary text data like item descriptions or reviews. However, existing methods face a trade-off: shallow models (e.g., LDA) are fast but ignore sentence structure, while deep learning models (e.g., RNNs, CNNs) capture context but are computationally expensive and complex to integrate.
The authors propose P2VMF, a recommendation model that bridges this gap. By integrating unsupervised paragraph embeddings into Probabilistic Matrix Factorization (PMF), the system captures deep semantic context from item descriptions to provide accurate recommendations even when user rating data is extremely sparse.
Methodology
The P2VMF architecture consists of two main pillars: Paragraph to Vector (P2V) and Probabilistic Matrix Factorization (PMF).
Paragraph to Vector (P2V)
This component is an extension of the Continuous Bag-of-Words (CBOW) and Sent2Vec models. Instead of just looking at individual words, it learns to represent entire paragraphs as fixed-length vectors. This allows the model to capture the semantic essence of item descriptions or reviews.
Integration with PMF
The core innovation lies in how these embeddings are used. The P2V output is not just a side-feature; it is used as the mean of the Gaussian distribution that generates the item's latent factors in the PMF model. This seamlessly connects the text's semantic meaning to the recommendation logic.
The authors developed three versions of the model:
- P2VMF: Treats sentences as independent (ignores order).
- P2VMF+: Incorporates sentence ordering to better capture flow and meaning.
- P2VMF++: Further refines the model by subdividing long reviews/descriptions into distinct paragraphs to capture more granular sentiment.
Experimental Results
The model was evaluated on three real-world datasets: MovieLens-1m, MovieLens-10m, and Amazon Instant Video (AIV). The AIV dataset is extremely sparse (0.005% density), making it a critical test for the model's ability to leverage text context.
Key findings include:
- Superior Performance on Sparse Data: On the AIV dataset, P2VMF++ achieved a Root Mean Squared Error (RMSE) of 1.1812, significantly outperforming standard PMF (1.4723) and even the deep-learning-based ConvMF+ (1.2014).
- Robustness: In tests on the ML-1m dataset where training data was reduced to only 20%, P2VMF+ maintained an RMSE of 0.9818, while the standard PMF spiked to 1.1009.
- Efficiency vs. Accuracy: By using an unsupervised "shallow" embedding approach, the authors achieved results comparable to or better than deep learning models while maintaining lower model complexity.
Conclusion
P2VMF demonstrates that paragraph embeddings are a powerful tool for overcoming data sparsity in recommendation systems. By capturing sentence order and granularity, the model provides a middle ground between simplistic bag-of-words models and heavy deep learning architectures. Future work will investigate cross-domain recommendations, using these embeddings to bridge user preferences across different item categories.
Summary updated