Abstract
Training video concept detectors typically requires vast amounts of manually labeled data, which is prohibitively expensive to acquire at scale. This paper introduces WELL (WEbly-Labeled Learning), a framework designed to learn directly from noisy web metadata such as titles and tags. By combining curriculum learning with a novel partial-order approach and sample-level dropout, WELL effectively mitigates the impact of label noise. Experimental results on the FCVID and YFCC100M datasets demonstrate that WELL achieves state-of-the-art performance, proving that large-scale noisy web data can effectively substitute for smaller, manually curated datasets.
Introduction
The "data hunger" of modern computer vision models presents a significant bottleneck for video analysis. While image datasets like ImageNet boast millions of labels, video datasets are often orders of magnitude smaller due to the high cost of manual annotation. For instance, the Fudan-Columbia Video Dataset (FCVID) contains only 0.09 million labels compared to ImageNet’s 14 million.
To address this, we propose leveraging the abundance of "web labels"—user-provided titles, descriptions, and tags. However, web data is notoriously noisy; keywords may be missing (low recall) or irrelevant to the visual content (low precision). The WELL framework provides a theoretically grounded approach to harness this messy data effectively.
Methodology: The WELL Framework
The core of WELL is built upon two pillars: Curriculum Learning (CL) and Self-Paced Learning (SPL). CL mimics human education by presenting "easy" examples before "hard" ones, while SPL allows the model to dynamically adjust its learning pace based on its current state.
Partial-Order Curriculum
Traditional curriculum learning requires a "total order," ranking every video by difficulty—a task that is computationally infeasible at web scale. WELL introduces a partial-order curriculum, where videos are grouped into broad categories based on metadata confidence (e.g., "keyword in title" vs. "keyword in tags"). The model prioritizes confident groups without requiring a strict internal ranking.
Sample-Level Dropout
To prevent the model from overfitting to specific noisy labels early in training, we introduce sample-level dropout. Similar to neural network dropout, this technique randomly ignores subsets of samples during iterations, encouraging the model to learn more robust, generalizable visual patterns rather than memorizing noise.
Experimental Results
We evaluated WELL on two massive benchmarks: FCVID (91,223 videos) and YFCC100M (~800,000 videos). In both cases, WELL significantly outperformed existing baselines.
On the FCVID dataset, WELL achieved a Mean Average Precision (mAP) of 0.567, a substantial improvement over BatchTrain (0.469) and BabyLearning (0.496). On the YFCC100M dataset, WELL maintained high precision (P@10 of 0.649), outperforming Google’s hard negative mining approach (0.500).
Discussion: Quantity vs. Quality
A key insight from this research is the trade-off between data volume and label quality. Our experiments show that a model trained on a large volume of noisy data can surpass one trained on a smaller set of clean data.
As shown in the table above, WELL trained on 2,000 hours of noisy web data achieved a performance of 0.567, which is superior to the 0.561 achieved by BatchTrain using 500 hours of manually labeled "clean" data. This suggests that for video concept detection, scale can compensate for noise when using a robust learning framework.
Conclusion and Future Work
WELL demonstrates that webly-labeled data is a viable and powerful resource for training video detectors. By formalizing the "easy-to-hard" heuristic as a non-convex optimization problem, we provide a solid theoretical foundation for large-scale web learning. Future work will focus on automating the "early stopping" process to remove the need for any manual validation and integrating WELL into end-to-end deep learning architectures for joint feature and detector optimization.
Summary updated