Abstract
This paper describes the participation of the DFKI-IUPR team in the TRECVID 2009 High-level Feature Extraction task. We investigate the utility of large-scale, weakly-labeled web video data from YouTube for training visual concept detectors intended for professional broadcast archives. To address the inherent domain shift between web and broadcast content, we employ a linear adaptation technique based on Passive Aggressive Online Learning (PAMIR). Our experiments demonstrate that while direct application of web-trained models to broadcast data yields poor results, a rapid adaptation step significantly enhances performance. Furthermore, the linear modeling approach provides a 500-fold training speedup compared to traditional non-linear Support Vector Machines (SVMs), facilitating efficient model iteration on massive datasets.
1. Introduction: The Domain Gap Challenge
In computer vision, the acquisition of expert-labeled training data is a significant bottleneck. While platforms like YouTube provide a nearly inexhaustible supply of video content, this data is often "noisy"—tags are subjective, and only a fraction of a video's duration may be relevant to its label.
A more fundamental challenge is the domain change problem. Visual concepts often manifest differently across different media types. For example, a "telephone" in a YouTube video is frequently a close-up of a mobile device (Figure 2), whereas in professional broadcast archives, it is more likely to be a desk phone in an office setting (Figure 1). This shift in visual characteristics often causes detectors trained on web data to fail when applied to professional TV archives.
2. Methodology
We compared several strategies for high-level feature extraction and model training:
2.1 Feature Extraction
- SIFT Visual Words: Local descriptors aggregated into a standard "bag-of-visual-words" representation.
- TubeTagger: A semantic feature space where frames are represented by scores from 233 pre-existing concept detectors (e.g., "street", "vehicle").
2.2 Statistical Modeling
- Support Vector Machines (SVMs): Utilizing a $\chi^2$ kernel, SVMs represent the state-of-the-art in accuracy but are computationally expensive for large datasets.
- Passive Aggressive Online Learning (PAMIR): A linear discriminative approach that optimizes the area under the ROC curve. PAMIR is significantly faster and supports efficient online updates.
2.3 Domain Adaptation
Our core approach involves training an initial model on YouTube data to establish a weight vector, followed by a secondary training sequence on a smaller set of annotated TRECVID data to adapt those weights to the target domain. This "on-line" adaptation is computationally inexpensive compared to retraining from scratch.
3. Experimental Setup
Two primary datasets were utilized:
- YOUTUBE: A custom dataset comprising approximately 120 hours of video. We collected 150 videos for each of 20 concepts using keyword queries and category filters.
- TRECVID '09: The standard "Sound & Vision" development data, featuring high-quality expert annotations for the annual competition.
4. Results and Discussion
Performance was evaluated using inferred Mean Average Precision (infMAP). The results highlight that adaptation is essential; the step from training only on YouTube (3.2% infMAP) to adapting that model to TRECVID (5.1% infMAP) yielded a significant performance boost. This proves that the model successfully transferred knowledge from the web data.
While the linear PAMIR approach did not surpass the non-linear SVM (which achieved 8.5% infMAP) in absolute accuracy, it offered a 500-fold speedup in training time. In the context of Big Data, this efficiency allows for much faster experimentation and iteration, which is often more valuable than marginal gains in precision.
5. Conclusion and Future Work
The primary limitation of the current approach is the performance gap between linear adaptation and non-linear SVMs. Additionally, the optimal number of adaptation iterations appears to be concept-dependent; some concepts are easily "over-adapted," while others require more training to bridge the domain gap. Future research will focus on per-concept adaptation rates and improved filtering techniques to remove "junk" frames from web-sourced training data.
Summary updated