Abstract
Researchers from National Taiwan University (NTU) developed a specialized toolkit that reduces SVM training time for video concept detection from two weeks to just three days. By combining optimized training with the "recycling" of past detectors and novel post-processing techniques—specifically concept reranking and temporal filtering—they achieved a 46% performance boost in High-Level Feature (HLF) detection.
Introduction
Semantic concept detection, the identification of high-level features such as "airplane," "sports," or "desert" in video shots, is fundamental to video indexing. However, the field faces significant challenges: high computational costs for training Support Vector Machines (SVMs), the scarcity of annotated data, and "context blindness" where models ignore temporal continuity and inter-concept relationships. This paper presents a three-pillar framework designed to overcome these hurdles through optimized training, model aggregation, and context-aware post-processing.
Methodology
The NTU framework consists of three primary components: an optimized toolkit, a model aggregation strategy, and a post-processing pipeline.
The NTU Toolkit
The team modified the standard LIBSVM library to optimize it specifically for high-level feature detection. Key improvements include:
- Dense Inner Products: Optimized code for dense feature vectors halved computational time.
- Parallelism: Simultaneous processing of multiple concept-feature pairs pushed CPU utilization above 90%.
- Narrowed Search: Reducing the parameter grid search from 110 to 42 points accelerated the process without compromising quality.
Post-processing and System Architecture
The post-processing framework exploits both inter-concept and inter-shot relationships to refine initial detections.
- Concept Reranking: This utilizes "inter-concept" relationships. For instance, high confidence in "Water" can be used to boost the score of a "Boat" detector in the same shot.
- Temporal Filtering: This addresses "inter-shot" relationships. Since concepts like "Weather" often span multiple shots, predictions are smoothed using a weighted combination of neighboring shots determined by statistical tests.
Aggregation of Past Detectors
Rather than training solely on new data, the authors employed "late aggregation" to combine TRECVID 2007 detectors with models trained on 2005–2006 data. They utilized Concept-dependent Weighted Aggregation, where weights are learned based on the performance of older detectors on a validation set.
Experimental Results
The framework was evaluated on the TRECVID 2007 benchmark, utilizing data from 2005 and 2006 as "past data." The NTU toolkit reduced training time from 14 days to just 3 days, enabling more iterative experimentation.
As shown in the results table, the best-submitted run (A_ntTransformer_5) achieved an infMAP of 0.0874, representing a 46% improvement over the baseline (A_nt20Giants_6). Specific gains were observed for contextually related concepts: reranking improved "Charts" by 45% and "Boat/Ship" by 29%, while temporal filtering boosted "Weather" detection by 37%.
Conclusion
The NTU framework demonstrates that significant gains in video concept detection can be achieved through efficiency and context. By "recycling" old models and applying temporal and contextual post-processing, the researchers improved accuracy without requiring massive new datasets. Future work will explore the parallel combination of reranking and filtering to further enhance performance.
Summary updated