Abstract
The UCF team’s entry for TRECVID 2009 addresses the challenge of detecting specific semantic concepts within vast, uncurated video datasets. By combining a Bag-of-Words (BoW) framework with a specialized bootstrapping technique, the approach mitigates the extreme imbalance between positive and negative training examples, effectively reducing false alarms in high-level feature extraction.
Introduction: The "Needle in a Haystack" Problem
High-level feature extraction (HLF) involves automatically detecting semantic concepts—such as "bus," "classroom," or "person singing"—within video shots. This task is complicated by two primary factors:
- The Semantic Gap: While computers easily detect low-level features like edges or colors, identifying complex concepts requires understanding the relationships between various objects and their contexts.
- Data Imbalance: In datasets like TRECVID, which contains hundreds of hours of video, a specific concept might only appear in a handful of shots. Training classifiers on such lopsided data often results in systems overwhelmed by noise, leading to excessive false positives.
Methodology
The UCF pipeline is built upon a refined Bag-of-Words model designed for robustness and efficiency.
Keyframe Extraction and Weighting
To represent video shots, the team used 3D HSV color histograms to cluster frames, selecting representative keyframes from the most stable clusters. They applied a Mean-TF*IDF² weighting scheme to SIFT descriptors. This approach assumes that visual "words" appearing frequently in a few images but rarely across the entire dataset are the most discriminative for classification.
Bootstrapping for Imbalance
The core innovation of the system is an iterative bootstrapping process. The training begins with all available positive examples and an equivalent number of negative ones. After training an initial Support Vector Machine (SVM) classifier, the system is tested against the remaining negative pool. The "hardest" negatives—those the system incorrectly identifies as positives—are added to the training set for the next iteration.
Experimental Setup and Results
The system was evaluated on the TRECVID 2009 dataset, consisting of approximately 280 hours of television broadcasts from the Netherlands Institute for Sound and Vision. The task involved detecting 20 specific features across roughly 300,000 extracted keyframes.
Performance Analysis
The bootstrapping method significantly improved the system's precision by reducing the Probability of False Alarm (PFA) by 26%. While this led to a 12% drop in the Probability of Detection (PD), the trade-off was deemed necessary to prevent the retrieval results from being dominated by false positives.
As shown in the results, the UCF system generally performed at the median level compared to other participants. The team also found that a single, common vocabulary of 1,000 visual words was as effective as feature-specific vocabularies, offering significant computational savings.
Discussion and Future Directions
The 2009 evaluation highlighted several areas for improvement. The reliance on static SIFT descriptors made it difficult to identify dynamic actions like "person riding a bicycle." Future iterations aim to incorporate optical flow to capture temporal information. Additionally, the team plans to move from binary visual word assignment to soft-assignment and explore color-invariant descriptors like opponentSIFT to better handle the lighting variability inherent in "live" broadcast video.
Summary updated