Introduction
Collaborative filtering systems typically operate on sparse user-item rating matrices. A common but often unexamined assumption in these systems is that data is Missing at Random (MAR). Under MAR, the probability that a rating is missing is independent of the rating's value. However, empirical evidence suggests that users do not rate items randomly; they primarily rate items they have chosen to consume based on prior preferences. This leads to a Missing Not at Random (MNAR) scenario, where selection bias significantly skews the observed data.
Methodology: Modeling the Observation Process
To address the MNAR problem, the authors propose incorporating an explicit model of the missing data process into the recommendation framework.
1. The Data Model
The base model is a Multinomial Mixture Model (MMM), which assumes users belong to latent classes with distinct preference distributions.
2. Missing Data Models
Two approaches are introduced to model the probability of an item being rated:
- CPT-v (Conditional Probability Table): This models the probability of a rating existing as a function of the rating value itself. It captures the "apathy effect," where users are more likely to rate items they feel strongly about (highly positive or highly negative) and less likely to rate neutral items.
- Logit-vd: This extends the model using a logistic function that incorporates both the rating value and item-specific popularity factors, accounting for the fact that famous items are more frequently rated regardless of individual preference.
Empirical Analysis of Selection Bias
The authors analyze several datasets to demonstrate the prevalence of selection bias. In datasets like EachMovie, MovieLens, and Netflix, ratings are heavily skewed toward higher values.
As shown in the YouTube dataset, nearly 90% of ratings are five stars. This suggests that users almost exclusively rate content they enjoy, leaving the system with little information about what they dislike. To validate their models, the authors utilized a unique Yahoo! Music dataset containing both user-selected ratings and ratings for items selected uniformly at random.
The distribution of random ratings reveals a starkly different reality: when forced to rate random items, users provide far more low ratings. Standard models trained on biased user-selected data fail to generalize to this "ground truth" distribution because they assume the missing data is representative of the whole.
Results and Discussion
The proposed MNAR-aware models (MM/Logit-vd and MM/CPT-v) were compared against standard baselines such as Matrix Factorization and Item-based K-Nearest Neighbors.
- Prediction Accuracy: On the unbiased Yahoo! Random test set, the MNAR models significantly outperformed standard MAR-based models. Standard models tended to over-predict high ratings, failing to account for the fact that unrated items are likely to be disliked.
- Ranking Performance: The MNAR models achieved higher Normalized Discounted Cumulative Gain (NDCG), effectively placing preferred items higher in recommendation lists by leveraging the "absence of evidence" as evidence of low preference.
Conclusion
This research highlights that the MAR assumption is a dangerous simplification for recommender systems. By explicitly modeling the missingness mechanism, we can mitigate selection bias and provide more robust, personalized recommendations. Future work includes exploring dependencies between missing ratings and incorporating auxiliary data like content features or social signals.
Summary updated