Abstract
The MESH consortium participated in the TRECVID 2008 High Level Feature Extraction (HLFE) task, deploying an automated pipeline for semantic concept detection in video. The approach explored various configurations, including MPEG-7 baselines, SIFT Bag-of-Words, and multi-stage classification strategies. Results highlighted the effectiveness of late fusion (combining SIFT and MPEG-7) but also revealed how a critical implementation bug in the baseline training phase can propagate through dependent models, underscoring the importance of robust validation in automated systems.
Introduction
High Level Feature Extraction (HLFE) involves the automatic detection of semantic concepts—such as "Cityscape," "Two People," or "Nighttime"—within video shots. For TRECVID 2008, the MESH consortium adapted its news-tuned visual analysis infrastructure to a broader concept set. A primary goal was to develop a "blind" training system capable of automatically selecting optimal models and feature configurations without human intervention.
Methodology
The MESH team submitted six distinct system configurations, or "runs," primarily based on a Support Vector Machine (SVM) architecture. These runs explored different feature sets and classification strategies:
- MPEG-7 Baseline (Run 4): Utilized global visual descriptors (e.g., color layout, edge histogram) with an RBF-kernel SVM.
- SIFT Bag of Words (Run 5): Employed a 10,000-word visual vocabulary derived from random SIFT keypoints.
- Visual Words & Part-Based Detection (Run 3): Combined Haar/Harris features for object parts with custom color-segmentation for scene concepts.
- 2-Stage Classification (Run 6): Aimed to capture concept correlations by feeding baseline predictions into a second-stage SVM.
- Extended 2-Stage (Run 1): Augmented the 2-stage setup with HOG descriptors and motion activity features.
- Combined SIFT & MPEG-7 (Run 2): A late-fusion approach merging global MPEG-7 and local SIFT features.
The system relied on TRECVID 2008 development data and collaborative annotations for training. Additionally, the PASCAL VOC2006 dataset was used to train part-based detectors for specific concepts in Run 3.
Results and Discussion
The experimental results provided a stark contrast between the potential of the architecture and the impact of implementation errors. Run 2, the late-fusion model, was the most successful, performing near the median of all TRECVID participants.
A significant finding was the "cascading bug" effect. An error in calculating Mean Average Precision (MAP) during the baseline training phase—specifically the omission of false negatives—led to severe overfitting. Since Runs 1, 3, and 6 were dependent on this baseline, their performance was significantly degraded. Despite this, the models maintained decent precision at the very top of ranked lists (e.g., 20% precision at 5 shots), suggesting they successfully identified obvious concept instances but failed on more complex, "long-tail" examples.
Conclusion
The MESH participation in TRECVID 2008 serves as both a demonstration of advanced fusion techniques and a cautionary tale regarding pipeline integrity. While the late-fusion approach (Run 2) proved resilient, the failure of dependent models due to a baseline bug highlights the necessity of rigorous cross-validation and error-checking in automated machine learning workflows. Future work will focus on correcting these optimization bugs, refining the SIFT Bag-of-Words approach, and developing better sampling strategies for highly unbalanced datasets.
Summary updated