Abstract
This paper details the University of Marburg's approach to the TRECVID 2008 high-level feature extraction task. The research focuses on mitigating extreme class imbalance in video datasets—where positive examples of concepts like "mountain" or "singing" are rare compared to negative ones. By implementing negative sub-sampling and augmenting positive training data through image variations, the team improved detection performance from a 5.91% baseline to a peak of 8.27% mean inferred average precision (mInAP).
Introduction
In video retrieval, high-level feature extraction involves training models to recognize complex semantic concepts within video shots. A persistent challenge is the "needle in a haystack" problem: for any specific concept, the dataset typically contains a vast majority of negative examples. Standard machine learning models, such as Support Vector Machines (SVMs), often become biased toward this majority class. This study investigates whether performance can be enhanced through strategic training data management rather than solely increasing feature complexity.
Methodology
The system utilizes a multi-modal framework that integrates low-level, mid-level, and audio features to classify video segments.
Feature Extraction
- Visual Features: Standard descriptors including color moments, correlograms, and Gabor wavelets were employed. The team introduced Hough features for geometric line detection and utilized image variations (rotation, zooming, smoothing) to augment the positive training set.
- Audio Features: A hierarchical SVM classification tree was used to categorize audio segments into speech, music, background noise, or action sounds.
- Mid-Level Features: The system identifies specific "anchors" such as faces, on-screen text, and camera motions (e.g., pans and tilts).
Training Strategy
The core of the experimentation involved six distinct "runs" to test different configurations. The most significant technique was negative sub-sampling, where only 25% of negative samples were used for training. This approach forces the SVM to establish a more effective decision boundary by reducing the overwhelming influence of the majority class.
Results
The evaluation used the inferred average precision (infAP) metric, which is suitable for large-scale retrieval tasks with incomplete ground truth.
The baseline performance (Marburg1) yielded 5.91% mInAP. The most successful configuration, Marburg2, which utilized negative sub-sampling, achieved a peak performance of 8.27% mInAP. Run Marburg5, which focused on data augmentation through image variations, reached 7.39% mInAP, demonstrating that synthetic positive samples can effectively combat class imbalance for certain concepts.
Discussion and Key Insights
The results highlight several critical aspects of high-level feature extraction:
- The Power of Sub-sampling: Discarding 75% of negative data led to the highest performance gains, suggesting that a "leaner" negative set is more beneficial for highly imbalanced tasks.
- Feature Bloat and Noise: The addition of Hough and audio features unexpectedly caused a slight decrease in performance. This illustrates the "curse of dimensionality," where additional data types introduce more noise than discriminative signal.
- Concept-Specific Augmentation: Image variations were highly effective for concepts with stable visual structures, such as "Mountain," but less so for others.
- Overfitting Risks: Attempts to use cross-validation for automated feature selection resulted in a performance drop to 4.07%, as the models over-optimized for the training data and failed to generalize.
Conclusion
The University of Marburg's participation in TRECVID 2008 underscores the importance of data balancing in video concept detection. While multi-modal features provide a foundation, the strategic handling of class imbalance through negative sub-sampling proved to be the most impactful factor in improving retrieval accuracy. Future work will explore more robust regularization techniques and sophisticated fusion methods to better leverage audio and geometric features.
Summary updated