BUPT at TRECVID 2008

Zan Gao, Zhi-Cheng Zhao, Tao Liu, Xiaoming Nan, Mei Mei, Bin Zhang, Xiaodan Liu, Xu Peng, Hui Zheng, Yanyun Zhao, Anni CaiTRECVID 2008 · OpenReview

Abstract

The Beijing University of Posts and Telecommunications (BUPT) team participated in the TRECVID 2008 challenge, focusing on High-Level Feature (HLF) extraction and Content-Based Copy Detection (CBCD). Their approach emphasized a "fusion-heavy" strategy, combining diverse visual descriptors like SIFT and Gabor wavelets with various SVM ensemble methods. This paper details their modular pipeline, the impact of different annotation styles, and the effectiveness of various fusion schemes in bridging the semantic gap and ensuring robust copy detection.

Introduction

As digital video libraries expanded rapidly in the late 2000s, the need for automated systems to understand and protect content became critical. The TRECVID 2008 challenge presented two primary hurdles: bridging the "semantic gap" in HLF extraction (translating raw pixels into concepts like "classroom" or "mountain") and achieving robustness in CBCD against transformations like resizing or "video-in-video" embedding. The BUPT team aimed to balance computational efficiency with retrieval accuracy within this standardized benchmarking framework.

Methodology

High-Level Feature (HLF) Extraction

The HLF extraction process relied on a modular pipeline that integrated multiple visual descriptors rather than a single feature.

  • Feature Engineering: The team extracted SIFT for local points, Gabor Wavelets for texture, Edge Orientation Histograms for shape, and various color-based features (RGB moments, HSV histograms).
  • Classification: Support Vector Machines (SVM) with Radial Basis Function (RBF) kernels were employed. Parallel computing was used to manage the high computational load of training.
  • Fusion Strategies: Four fusion schemes were evaluated to combine classifier outputs: Maximum, Average, Voting, and Linear Weighted (based on precision).
The modular pipeline for High-Level Feature (HLF) extraction, showcasing the fusion of multiple visual descriptors.

Content-Based Copy Detection (CBCD)

The CBCD system was designed to identify video copies even under significant distortion.

  • Video Segmentation: A "difference of frame difference" method was introduced to segment videos into sub-clips. This approach effectively ignored gradual transitions that often cause errors in simpler algorithms.
  • Tiered Search: The system utilized different strategies based on query length. For longer clips, the search began from the second segment to account for potential missing frames at the boundaries of copies.
The Content-Based Copy Detection (CBCD) workflow, featuring video segmentation and tiered search strategies.

Experimental Setup and Datasets

The study utilized the TRECVID 2008 dataset. For HLF training, three annotation styles were tested on the TRECVID 2007 development corpus:

  1. Annotation-I: Standard provided annotations.
  2. Annotation-II: BUPT's re-annotation using bounding boxes for region-related concepts.
  3. Annotation-III: Whole-frame annotation using the IBM MPEG-7 tool, regardless of concept type.

Results and Discussion

The team's best HLF run achieved a Mean Average Precision (MAP) of 0.047, performing slightly above the participant average for that year. A key insight was that Annotation-III (whole frame) outperformed specific bounding boxes, suggesting that surrounding context is highly informative for SVM classifiers. Furthermore, Maximum Fusion proved most effective for certain concepts, indicating that the most confident classifier often provides the best signal.

In CBCD, the system achieved high precision and a strong Mean F1 score. However, it struggled with "video-in-video" transformations, where the copy is a small portion of a larger frame, highlighting a need for more robust local features.

Conclusion

The BUPT team's participation in TRECVID 2008 demonstrated that data annotation and model fusion are as critical as feature selection. Future work will focus on improving robustness against complex transformations like "video-in-video," enhancing SVM training for high-dimensional data, and optimizing indexing structures to increase retrieval speed for real-world applications.

Summary updated

BibTeX
@inproceedings{gao2008bupt,
  author = {Zan Gao and Zhi-Cheng Zhao and Tao Liu and Xiaoming Nan and Mei Mei and Bin Zhang and Xiaodan Liu and Xu Peng and Hui Zheng and Yanyun Zhao and Anni Cai},
  title = {{BUPT at TRECVID 2008}},
  booktitle = {TRECVID},
  year = {2008},
  url = {https://openreview.net/forum?id=p3nIDnambsk},
}
Figure