Nanjing University in TRECVID 2009

Yang Yang, Jingwei Xiao, Kang Lin, Gangshan Wu, Tongwei Ren, Yaqiong Wang

Introduction

Content-Based Copy Detection (CBCD) has become a critical necessity as digital video libraries expand. The goal is to identify video copies even after significant modifications such as flipping, resizing, or re-encoding. This paper documents Nanjing University’s inaugural participation in the TRECVID 2009 Video Copy Detection task, exploring the trade-offs between high-accuracy local feature indexing and memory-efficient global representations.

Methodology

The research team implemented two distinct pipelines to address the "video-only" and "video + audio" tasks, centered on Speeded Up Robust Features (SURF).

Visual Feature Extraction

The team utilized "Upright" SURF (U-SURF) descriptors, as the benchmark transformations did not include rotation. Keyframes were sampled at approximately 2 frames per second (fps), specifically targeting I-frames to streamline the decoding process.

Indexing Strategies

Two primary indexing methods were tested:

  1. Direct Local Feature Indexing (Video-only): This approach utilized Locality-Sensitive Hashing (LSH) via the E2LSH library. To account for horizontal mirroring, the team mathematically transformed SURF descriptors during the query phase rather than doubling the database size.
  2. Bag-of-Features (BoF) Approach (Video + Audio): To mitigate the memory intensity of LSH, the team transitioned to a BoF model. SURF descriptors were clustered into a visual vocabulary using K-Means, representing each frame as a histogram. These histograms were indexed using Random KD-trees via the FLANN library.
Figure 1: Pipeline for the video-only task utilizing direct feature indexing.
Figure 2: Pipeline for the video + audio task utilizing a Bag-of-Features (BoF) approach.

Challenges and Insights

The participation highlighted significant engineering hurdles in scaling computer vision algorithms:

  • The Memory Wall: LSH proved extremely resource-heavy. While theoretical models suggested 160 hash tables for optimal performance with 64-dimensional SURF features, hardware limitations (a 5-PC cluster with 4GB RAM each) forced a reduction to only 10 hash tables.
  • Dimensionality Reduction: To further save memory, descriptors were reduced from 64 to 16 dimensions, which likely degraded retrieval accuracy.
  • System Integration Issues: Technical failures in the MPEG audio decoder prevented the inclusion of audio features in the multi-modal runs. Additionally, the inability to run official sample generators on local hardware meant the system was submitted without a refinement phase.

Results and Discussion

The performance was characterized by the authors as "disappointing," primarily due to the aforementioned constraints. In the video-only task, the insufficient number of hash tables led to poor retrieval rates. In the BoF-based runs, sparse sampling (2 fps) and suboptimal parameter tuning for the FLANN index hindered accuracy.

Conclusion

Nanjing University’s first TRECVID entry serves as a valuable case study in the practical limitations of high-dimensional indexing. Future work will focus on optimizing memory efficiency for LSH and improving the robustness of multi-modal integration to overcome the "memory wall" inherent in large-scale video analysis.

Summary updated

BibTeX
@inproceedings{yang2009nanjing,
  author = {Yang Yang and Jingwei Xiao and Kang Lin and Gangshan Wu and Tongwei Ren and Yaqiong Wang},
  title = {{Nanjing University in TRECVID 2009}},
  booktitle = {TRECVID},
  year = {2009},
}
Figure