Introduction
In the mid-2000s, Information Retrieval (IR) and Recommender Systems were largely viewed as separate fields. However, researchers Masao Utiyama and Mikio Yamamoto proposed a unifying perspective: recommendation can be framed as a search problem where a user's history acts as a query, and "relevance feedback" is used to find similar items. Their work introduces the Polya model, a language modeling approach that effectively combines collaborative filtering (CF) and content-based filtering (CBF) while addressing the "burstiness" inherent in human behavior.
Addressing the "Burstiness" of User Behavior
Traditional recommendation models often struggle with repeated occurrences of features. In standard multinomial language models, if a user interacts with a feature twice, its probability is squared. In reality, the second interaction often provides less novel information than the first—a phenomenon known as "burstiness."
To account for this, the authors utilize an approximated Polya distribution (Dirichlet Compound Multinomial). The key innovation is a "dumped count" function, $\nu(n, \alpha)$, which squashes raw counts $n$ based on a parameter $\alpha$.
As shown in the figure above, when $\alpha$ is large, the function behaves like a standard linear count. When $\alpha$ is small, the function plateaus, effectively treating the feature as a binary indicator. This allows the model to recognize the importance of a user's interaction without letting a single high-frequency user or keyword dominate an item's profile.
Experimental Results
The researchers evaluated the Polya model across three datasets: English Wikipedia (edit histories), MovieLens (movie ratings), and BookCrossing (book ratings).
A primary finding was the significant superiority of the Polya model over the standard multinomial model (represented by a very large $\alpha$). On the Wikipedia dataset, the Polya model's ability to "dump" counts led to substantial gains in R-precision.
The results indicate that the Polya model improved R-precision by 3.4% for content-based filtering and a remarkable 17.4% for collaborative filtering. Furthermore, the hybrid model (CF+CBF) consistently outperformed individual components, demonstrating that user behavior and item content provide complementary signals.
Conclusion
By treating recommendation as an IR relevance feedback problem, this work successfully imports robust probabilistic frameworks into the recommendation domain. The Polya model's handling of bursty data proves crucial for both content and collaborative signals. While parameter tuning remains a challenge, the model's performance—matching or exceeding state-of-the-art baselines like CProb—highlights the potential of language modeling in building more nuanced recommender systems.
Reference: Masao Utiyama and Mikio Yamamoto.
- Relevance Feedback Models for Recommendation. In Proceedings of EMNLP 2006, pages 449–456.
Summary updated