Introduction
Content-Based Copy Detection (CBCD) has become a critical necessity as digital video libraries expand. The goal is to identify video copies even after significant modifications such as flipping, resizing, or re-encoding. This paper documents Nanjing University’s inaugural participation in the TRECVID 2009 Video Copy Detection task, exploring the trade-offs between high-accuracy local feature indexing and memory-efficient global representations.
Methodology
The research team implemented two distinct pipelines to address the "video-only" and "video + audio" tasks, centered on Speeded Up Robust Features (SURF).
Visual Feature Extraction
The team utilized "Upright" SURF (U-SURF) descriptors, as the benchmark transformations did not include rotation. Keyframes were sampled at approximately 2 frames per second (fps), specifically targeting I-frames to streamline the decoding process.
Indexing Strategies
Two primary indexing methods were tested:
- Direct Local Feature Indexing (Video-only): This approach utilized Locality-Sensitive Hashing (LSH) via the E2LSH library. To account for horizontal mirroring, the team mathematically transformed SURF descriptors during the query phase rather than doubling the database size.
- Bag-of-Features (BoF) Approach (Video + Audio): To mitigate the memory intensity of LSH, the team transitioned to a BoF model. SURF descriptors were clustered into a visual vocabulary using K-Means, representing each frame as a histogram. These histograms were indexed using Random KD-trees via the FLANN library.
Challenges and Insights
The participation highlighted significant engineering hurdles in scaling computer vision algorithms:
- The Memory Wall: LSH proved extremely resource-heavy. While theoretical models suggested 160 hash tables for optimal performance with 64-dimensional SURF features, hardware limitations (a 5-PC cluster with 4GB RAM each) forced a reduction to only 10 hash tables.
- Dimensionality Reduction: To further save memory, descriptors were reduced from 64 to 16 dimensions, which likely degraded retrieval accuracy.
- System Integration Issues: Technical failures in the MPEG audio decoder prevented the inclusion of audio features in the multi-modal runs. Additionally, the inability to run official sample generators on local hardware meant the system was submitted without a refinement phase.
Results and Discussion
The performance was characterized by the authors as "disappointing," primarily due to the aforementioned constraints. In the video-only task, the insufficient number of hash tables led to poor retrieval rates. In the BoF-based runs, sparse sampling (2 fps) and suboptimal parameter tuning for the FLANN index hindered accuracy.
Conclusion
Nanjing University’s first TRECVID entry serves as a valuable case study in the practical limitations of high-dimensional indexing. Future work will focus on optimizing memory efficiency for LSH and improving the robustness of multi-modal integration to overcome the "memory wall" inherent in large-scale video analysis.
Summary updated