TL;DR
This paper tackles the challenge of expensive video labeling by leveraging YouTube's "co-watch" data to build a video classification system. They propose a graph-based approach to select weakly labeled videos, demonstrating competitive performance against state-of-the-art methods, especially when manually labeled data is scarce.
What Problem Does This Paper Address?
Classifying videos on the web is crucial for applications like search and targeted advertising. However, the biggest hurdle is the prohibitive cost of collecting large amounts of manually labeled video data. While semi-supervised learning offers a solution by using inexpensive, albeit noisy, weakly labeled data, effectively selecting this data is key. This paper focuses on how to exploit the implicit social behavior of YouTube viewers – specifically, which videos they watch in sequence (co-watch data) – to improve video classification accuracy, particularly in scenarios with limited manually labeled data.
Methods
The core of the paper's contribution lies in its novel approach to selecting weakly labeled training data. They construct a video co-watch graph where nodes represent videos and edges signify co-watch relationships. This graph is then subjected to local graph clustering to identify relevant videos for training.
Here's a breakdown of their method:
- Graph Construction: Edges are formed between videos if a significant number of users watch them in the same session. This captures implicit relationships between videos.
- Seed Videos: A small set of manually labeled videos (seeds) are used to initiate the clustering process.
- Local Graph Clustering: They adapt Andersen's dense partitioning algorithm. This algorithm iteratively expands a cluster around the seed videos by adding co-watched videos. Edge weights are assigned based on co-watch frequency and rank, propagating from the seeds.
- Pruning and Filtering: To combat noise, the clusters are pruned. Videos with low weights (indicating weaker relationships to the cluster) are removed. Crucially, they also incorporate a text classifier to filter out videos that are unlikely to belong to the category, further reducing label noise. This text classifier is trained on manually labeled web documents, leveraging readily available text data.
- Hierarchical Classification: The system uses a hierarchical one-against-all approach, training a binary classifier for each category in a taxonomy. Positive samples include the category and its descendants, while negative samples exclude ancestors.
Datasets
- YouTube Video Data: The primary source of data is YouTube, with a pool of approximately 24 million videos.
- Manually Labeled Videos: A dataset of 10,528 manually labeled videos was collected over a year for evaluation and as seed data for the co-watch clustering. These labels are considered ground truth.
- Manually Labeled Web Documents: A large corpus of web documents, manually labeled according to the same taxonomy, was used to train initial text-based classifiers.
Key Insights and Ideas
- Social Behavior as a Proxy for Similarity: The paper strongly validates the intuition that users tend to watch similar videos in sequence. This co-watch behavior is a powerful, albeit noisy, signal for video relatedness.
- Graph Clustering for Noise Reduction: Instead of simply taking immediate co-watched videos, graph clustering allows for a more principled way to discover related videos across multiple hops, while simultaneously providing a mechanism to prune noisy connections.
- Leveraging Text Classifiers for Video Data Selection: Integrating a text classifier (trained on web documents) into the graph clustering process is a clever way to filter out irrelevant videos from the co-watch graph, significantly improving the quality of the weakly labeled training data. This highlights the synergy between different data modalities.
- Competitive Performance with Limited Labeled Data: The most significant insight is that their approach can achieve performance comparable to classifiers trained on expensive, manually labeled data, especially when the amount of manual labels is limited. This has direct implications for building practical video classification systems.
Key Results
- Classifiers trained using the proposed co-watch graph clustering method achieved competitive performance compared to classifiers trained on manually labeled data, particularly for categories with fewer than 100 manually labeled videos.
- In scenarios with limited manually labeled data (10-100 videos), their automatically collected videos trained superior models in 92 categories and inferior models in 88 categories compared to direct text classification.
- The "Max" performance (best performing classifier from different weakly labeled sources) consistently showed better performance at most depth levels in the taxonomy compared to direct text classification.
- A positive correlation was observed between the seed set size and classifier performance, indicating that more initial manually labeled data leads to better results.
- The paper also found that a smaller training set size (150 videos) for co-watched videos led to higher precision but lower recall compared to a larger set (250 videos), suggesting a trade-off between label noise and data diversity.
Limitations and Future Work
- Label Noise Reduction: The authors acknowledge that reducing label noise in automatically collected training videos remains an area for future improvement.
- Bias in Weak Classifiers: They note that using text classifiers for noise reduction might introduce bias if the text classifiers themselves are biased.
- Seed Set Size: While a positive correlation was found, the paper mentions that the graph is truncated to show only categories with up to 100 seed videos, implying that exploring larger seed sets could be beneficial.
- Future Directions: The paper suggests exploring methods to further reduce label noise in the automatically collected training videos.
Summary updated