Keyword-Based TV Program Recommendation

Christian Wartena, Wout Slakhorst, Martin Wibbels, Zeno Gantner, Christoph Freudenthaler, Chris Newell, Lars Schmidt-Thieme

Abstract

Content-based recommendation systems often struggle with the "vocabulary mismatch" problem, where items with similar themes are described using different terminology. This paper proposes a novel approach that leverages keywords as an intermediate representation, enhanced by Markov chain smoothing to capture semantic relationships. By modeling keyword co-occurrences, the system develops a similarity measure that transcends exact string matching. Evaluation on a real-world BBC dataset demonstrates that this method can outperform traditional collaborative filtering techniques in specific domains.

Introduction

The "Cold Start" problem remains a significant challenge in recommendation systems, particularly for new content in streaming and live TV broadcasting where collaborative filtering (CF) lacks sufficient user interaction data. While content-based recommendation is the standard alternative, it typically relies on exact keyword matches using measures like Jaccard or Cosine similarity. This leads to the "vocabulary mismatch" problem: two programs about the same topic (e.g., "cinema" vs. "movies") may be treated as unrelated. This research aims to bridge this semantic gap by developing a recommendation logic that understands keyword relationships through distributional context.

Methods

The proposed approach refines item similarity calculation through three primary stages:

  1. Keyword Extraction: The authors compare standard tf.idf with a co-occurrence-based technique. The latter identifies keywords by analyzing how closely their distribution across the corpus matches the specific document's distribution.
  2. Markov Chain Smoothing: To address vocabulary mismatch, a Markov chain is defined over a bipartite graph of items and keywords. A three-step transition (item → keyword → item → keyword) generates a "smoothed" keyword distribution. This distribution includes not just the original keywords but also semantically related "neighboring" terms that appear in similar contexts.
  3. Similarity Measure: Smoothed distributions are compared using Jensen-Shannon Divergence (JSD). This allows the system to recognize similarity between items that share related concepts even if they share no identical keywords.

Experimental Setup

The methodology was evaluated on two distinct datasets: the BBC Broadcast Data and a MovieLens/IMDB subset. The BBC data includes editorial synopses and related web text, providing a rich context for keyword extraction.

Table 1: Statistics for the BBC and MovieLens datasets used in the evaluation.

Results and Discussion

On the BBC dataset, the keyword-based approach demonstrated significant efficacy. Using JSD on smoothed distributions consistently outperformed standard Jaccard coefficients on raw keyword sets. Notably, the model achieved a Root Mean Square Error (RMSE) of 0.283, surpassing a state-of-the-art Matrix Factorization (CF) model which scored 0.291.

Table 2: RMSE results for the BBC dataset across different data sources and distance measures.

However, results were domain-dependent. While successful for TV programs, the approach failed to beat a simple "user average" baseline for the MovieLens dataset. This suggests that while TV viewing habits are often topic-driven (and thus well-captured by keywords), movie preferences may rely more on attributes like actors, directors, or stylistic "vibe" that plot-based keywords do not fully represent.

Conclusion

This research highlights the power of using keywords as an abstraction layer for recommendation, particularly when enhanced by Markov chain smoothing to capture semantic context. While highly effective for topic-driven content like TV broadcasts, the findings underscore that the choice of content representation must align with the underlying drivers of user preference in a given domain. Future work will explore integrating broader metadata, such as genres and cast information, into the Markov chain model.

Summary updated

Abstract

Notwithstanding the success of collaborative filtering algorithms for item recommendation there are still situations in which there is a need for content-based recommendation, especially in new-item scenarios, e.g. in streaming broadcasting. Since video content is hard to analyze we use documents describing the videos to compute item similarities. We do not use the descriptions directly, but use their keywords as an intermediate level of representation. We argue that a nearest-neighbor approach relying on unrestricted keywords deserves a special definition of similarity that also takes word similarities into account. We define such a similarity measure as a divergence measure of smoothed keyword distributions. The smoothing is done on the basis of co-occurrence probabilities of the present keywords. Thus co-occurrence similarity of words is also taken into account. We have evaluated keyboard-based recommendations with a dataset collected by the BBC and on a subset of the MovieLens dataset augmented with plot descriptions from IMDB. Our main conclusions are (1) that keyword-based rating predictions can be very effective for some types of items, and (2) that rating predictions are significantly better if we do not only take into account the overlap of keywords between two documents, but also the mutual similarities between keywords.
BibTeX
@inproceedings{wartena2011keyword,
  author = {Christian Wartena and Wout Slakhorst and Martin Wibbels and Zeno Gantner and Christoph Freudenthaler and Chris Newell and Lars Schmidt-Thieme},
  title = {{Keyword-Based TV Program Recommendation}},
  booktitle = {ITWP @ IJCAI},
  year = {2011},
}
Figure