Introduction
The TRECVID 2009 campaign presented significant challenges in video analysis across three primary domains: High-Level Feature Extraction (HLFE), Automatic Video Search, and Content-Based Copy Detection (CBCD). This paper details the VIREO/DVMM team's participation, highlighting the introduction of Domain Adaptive Semantic Diffusion (DASD) to address the limitations of isolated concept detection and the use of adaptive fusion and geometric verification for search and copy detection tasks.
High-Level Feature Extraction via DASD
A core challenge in HLFE is the "domain shift" problem, where semantic relationships between concepts (e.g., "road" and "vehicle") vary across different video genres. To address this, the authors proposed Domain Adaptive Semantic Diffusion (DASD).
DASD models concepts as nodes in a graph, with edges representing their statistical affinity. Unlike static models, DASD employs a diffusion process to refine detection scores based on contextual evidence. For instance, a high-confidence detection of a "vehicle" can boost the score of a nearby, uncertain "road" detection. Crucially, the system adapts the graph's geometry online based on the test data, ensuring that the modeled relationships remain relevant to the specific video domain being processed.
Automatic Video Search and Flickr Context
For automatic search, the team moved beyond fixed-weight fusion of text and visual modalities. They implemented a "simulated query" approach to dynamically estimate the reliability of different search paths for specific query types.
A key innovation was leveraging external knowledge from Flickr. By analyzing metadata from 3.5 billion images, the team estimated concept affinities using Normalized Google Distance (NGD). This "Flickr Context Similarity" (FCS) provided a robust proxy for visual "common sense," outperforming purely linguistic ontologies like WordNet by capturing real-world visual co-occurrences.
Content-Based Copy Detection (CBCD)
The CBCD system utilized a Bag-of-Visual-Words (BoW) framework with SIFT features. To maintain high precision against transformations like cropping or re-encoding, the authors applied Enhanced Weak Geometric Consistency (EWGC) and a 2D Hough Transform. These techniques ensure that matched features between a query and a reference video maintain consistent spatial and temporal alignments, effectively filtering out false positives in large-scale datasets.
Experimental Results
Evaluation on the TRECVID 2009 benchmark demonstrated the efficacy of these approaches. In the HLFE task, DASD improved the Mean Average Precision (MAP) from a baseline of 0.156 to 0.163, showing gains in 17 out of 20 concepts.
In the search task, adaptive fusion yielded a 10% improvement over single-modality baselines. The CBCD system also achieved competitive results, particularly in re-encoding transformations (T3), reaching an Optimal Normalized Detection Cost Rate (Opt.NDCR) of 0.022.
Conclusion
The VIREO/DVMM participation in TRECVID 2009 underscores the importance of context and domain adaptation in video intelligence. While DASD and Flickr-driven search provided significant performance boosts, future work remains to improve localization accuracy in copy detection and to expand the vocabulary of reliable concept detectors to further bridge the semantic gap in video retrieval.
Summary updated