BUPT-MCPRL at TRECVID 2009

Zhicheng Zhao, Yanyun Zhao, Zan Gao, Xiaoming Nan, Mei Mei, Hui Zhang, Heng Chen, Xu Peng, Yuanbo Chen, Junfang Guo, Anni CaiTRECVID 2009 · OpenReview

Introduction

Video retrieval remains a significant challenge due to the "semantic gap"—the disconnect between low-level visual data (pixels, edges, colors) and high-level human concepts (e.g., "a person running" or "a ship in the water"). The BUPT-MCPRL team participated in TRECVID 2009, a premier international benchmark, to test their systems across four key domains: Automatic Search, High-Level Feature (HLF) Extraction, Copy Detection, and Event Detection.

Methodology

The team employed a variety of sophisticated techniques tailored to each task, emphasizing the integration of semantic similarity and multimodal fusion.

Automatic Search and Retrieval Architecture

The core of the search system utilized a Weight Distribution based on Semantic Similarity (WDSS) strategy. This involves mapping text queries to a predefined lexicon of concepts using WordNet to calculate similarity scores. They also explored "negative WDSS" to filter out irrelevant results by negatively weighting contradictory concepts.

System architecture for multi-modal video retrieval.

As illustrated in Figure 1, the retrieval process begins with query pre-processing, followed by three parallel retrieval paths: HLF-based, Concept-based (mapping text and examples to concepts), and Visual-based retrieval. These results are then combined via multimodal fusion and refined through re-ranking based on face information.

HLF Extraction and Copy Detection

For HLF extraction, the system used a "late fusion" approach, combining global features (Gabor wavelets, Color Histograms) with local features (SIFT, SURF, ASIFT, MSER). Adaboost was employed for feature selection to reduce dimensionality, while SVMs handled classification. Copy detection was handled by two parallel systems: one using SIFT features with a visual vocabulary and another using SURF features with Hamming embedding and the Smith-Waterman algorithm for temporal alignment.

Key Results and Insights

The BUPT-MCPRL team achieved significant success, particularly in the automatic search task, where they outperformed all other participants.

Their best run (MCPR4) achieved an infMAP of 0.131, ranking #1 among all submitted runs in the TRECVID 2009 automatic search task.

Average precision of the MCPR4 run compared to median and best scores across TRECVID topics.

Figure 3 demonstrates the performance of their run across various topics. The system consistently performed above the median and reached the "best" performance level for several specific topic numbers.

Key Insights

  • Text-Concept Mapping Wins: Mapping text queries to semantic concepts performed significantly better than searching based purely on visual examples.
  • Lexicon Size Matters: Combining multiple concept lexicons (HLF concepts + previous search topics) led to their best search performance, suggesting that a broader "vocabulary" helps bridge the semantic gap.
  • Failure of Naive Re-ranking: Attempting to re-rank search results based on face detection actually decreased performance due to the low precision of the face detection algorithm, proving that additional data is only helpful if it is high-quality.

Conclusion and Future Work

While the search and HLF tasks showed strong results, event detection remained "not satisfactory," highlighting the extreme difficulty of temporal action recognition in 2009. The authors acknowledged that future improvements must focus on more robust, high-precision face and person detection modules and more advanced machine learning techniques for complex real-world events.

Summary updated

BibTeX
@inproceedings{zhao2009bupt,
  author = {Zhicheng Zhao and Yanyun Zhao and Zan Gao and Xiaoming Nan and Mei Mei and Hui Zhang and Heng Chen and Xu Peng and Yuanbo Chen and Junfang Guo and Anni Cai},
  title = {{BUPT-MCPRL at TRECVID 2009}},
  booktitle = {TRECVID},
  year = {2009},
  url = {https://openreview.net/forum?id=bHjQ3nzEPlG},
}
Figure