Abstract
Content-based recommendation systems often struggle with the "vocabulary mismatch" problem, where items with similar themes are described using different terminology. This paper proposes a novel approach that leverages keywords as an intermediate representation, enhanced by Markov chain smoothing to capture semantic relationships. By modeling keyword co-occurrences, the system develops a similarity measure that transcends exact string matching. Evaluation on a real-world BBC dataset demonstrates that this method can outperform traditional collaborative filtering techniques in specific domains.
Introduction
The "Cold Start" problem remains a significant challenge in recommendation systems, particularly for new content in streaming and live TV broadcasting where collaborative filtering (CF) lacks sufficient user interaction data. While content-based recommendation is the standard alternative, it typically relies on exact keyword matches using measures like Jaccard or Cosine similarity. This leads to the "vocabulary mismatch" problem: two programs about the same topic (e.g., "cinema" vs. "movies") may be treated as unrelated. This research aims to bridge this semantic gap by developing a recommendation logic that understands keyword relationships through distributional context.
Methods
The proposed approach refines item similarity calculation through three primary stages:
- Keyword Extraction: The authors compare standard tf.idf with a co-occurrence-based technique. The latter identifies keywords by analyzing how closely their distribution across the corpus matches the specific document's distribution.
- Markov Chain Smoothing: To address vocabulary mismatch, a Markov chain is defined over a bipartite graph of items and keywords. A three-step transition (item → keyword → item → keyword) generates a "smoothed" keyword distribution. This distribution includes not just the original keywords but also semantically related "neighboring" terms that appear in similar contexts.
- Similarity Measure: Smoothed distributions are compared using Jensen-Shannon Divergence (JSD). This allows the system to recognize similarity between items that share related concepts even if they share no identical keywords.
Experimental Setup
The methodology was evaluated on two distinct datasets: the BBC Broadcast Data and a MovieLens/IMDB subset. The BBC data includes editorial synopses and related web text, providing a rich context for keyword extraction.
Results and Discussion
On the BBC dataset, the keyword-based approach demonstrated significant efficacy. Using JSD on smoothed distributions consistently outperformed standard Jaccard coefficients on raw keyword sets. Notably, the model achieved a Root Mean Square Error (RMSE) of 0.283, surpassing a state-of-the-art Matrix Factorization (CF) model which scored 0.291.
However, results were domain-dependent. While successful for TV programs, the approach failed to beat a simple "user average" baseline for the MovieLens dataset. This suggests that while TV viewing habits are often topic-driven (and thus well-captured by keywords), movie preferences may rely more on attributes like actors, directors, or stylistic "vibe" that plot-based keywords do not fully represent.
Conclusion
This research highlights the power of using keywords as an abstraction layer for recommendation, particularly when enhanced by Markov chain smoothing to capture semantic context. While highly effective for topic-driven content like TV broadcasts, the findings underscore that the choice of content representation must align with the underlying drivers of user preference in a given domain. Future work will explore integrating broader metadata, such as genres and cast information, into the Markov chain model.
Summary updated