Multi-View Active Learning for Video Recommendation

Jia-Jia Cai, Jun Tang, Qing-Guo Chen, Yao Hu, Xiaobo Wang, Sheng-Jun Huang

Abstract

Modern video recommendation systems rely heavily on text metadata, which is frequently absent in user-generated content. This paper introduces Multi-View Active Learning (MVAL), a framework designed to mitigate the impact of missing features. MVAL utilizes a Visual-to-Text (V2T) mapping to synthesize missing text features from visual content and employs an active learning strategy to prioritize the most informative videos for human annotation. Experimental results on real-world datasets like YouTube and Youku demonstrate that MVAL significantly outperforms traditional methods, achieving higher accuracy with fewer labeled samples.

Introduction

Video platforms face a significant challenge: while raw visual data is always available, the high-level text metadata (titles, tags, descriptions) necessary for effective recommendation is often missing or sparse. Bridging this "semantic gap" manually is prohibitively expensive. Traditional active learning focuses on querying labels for unlabeled data. However, in recommendation systems, user actions (labels) are often abundant, while the descriptive features of the items are the scarce resource. MVAL addresses this by querying missing features rather than labels.

The MVAL Framework

The MVAL framework consists of two primary components: a Visual-to-Text (V2T) mapping module and an active selection strategy.

Visual-to-Text (V2T) Mapping

To handle missing text features, MVAL learns a mapping from the visual view to the text view. Unlike simple reconstruction, MVAL uses joint optimization. The model is trained to minimize both the difference between predicted and actual text features and the classification error of the recommendation model itself. This ensures that the synthesized text features are optimized for the end goal: predicting user interest.

The MVAL architecture illustrating the V2T mapping and joint optimization for classification.

Active Selection Strategy

Instead of random sampling, MVAL uses an informativeness score to select videos for annotation. This score is based on:

  • Prediction Inconsistency: The discrepancy between the model's prediction (using synthesized features) and the actual user action.
  • Watching Frequency: Prioritizing popular videos that act as "hubs" in the interaction graph, providing a stronger training signal.
  • Cost-Awareness: Adjusting the selection based on the effort required for annotation, such as video length.

Experimental Evaluation

The researchers evaluated MVAL across diverse tasks, including multi-view classification on YouTube and Wikipedia datasets, and real-world video recommendation using large-scale data from Youku (over 243,000 interactions).

Accuracy vs. number of queries on the YouTube dataset, comparing MVAL variants against random and uncertainty-based baselines.

As shown in the results for the YouTube dataset, MVAL-inconsistency (the red solid line) achieves significantly higher accuracy with fewer queries compared to random selection and standard uncertainty-based active learning. On the Youku datasets, MVAL reached performance levels with 1,000 queries that random selection required 5,000 queries to match, representing a 5x improvement in labeling efficiency.

Conclusion

Multi-View Active Learning provides a robust solution for recommendation systems plagued by missing metadata. By treating feature acquisition as an active learning problem and leveraging visual content to bridge the semantic gap, MVAL reduces the cost of building high-performance recommenders. Future work may explore more complex selection strategies and further reduce the reliance on initial seed data.

Summary updated

Abstract

On many video websites, the recommendation is implemented as a prediction problem of video-user pairs, where the videos are represented by text features extracted from the metadata. However, the metadata is manually annotated by users and is usually missing for online videos. To train an effective recommender system with lower annotation cost, we propose an active learning approach to fully exploit the visual view of videos, while querying as few annotations as possible from the text view. On one hand, a joint model is proposed to learn the mapping from visual view to text view by simultaneously aligning the two views and minimizing the classification loss. On the other hand, a novel strategy based on prediction inconsistency and watching frequency is proposed to actively select the most important videos for metadata querying. Experiments on both classification datasets and real video recommendation tasks validate that the proposed approach can significantly reduce the annotation cost.
BibTeX
@inproceedings{cai2019multi,
  author = {Jia-Jia Cai and Jun Tang and Qing-Guo Chen and Yao Hu and Xiaobo Wang and Sheng-Jun Huang},
  title = {{Multi-View Active Learning for Video Recommendation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/284},
}
Figure