Glasgow University at TRECVID 2007

P. Punitha, Joemon M. JoseTRECVID 2007 · OpenReview

Introduction

The "semantic gap"—the disconnect between low-level visual features and high-level human concepts—remains a primary hurdle in video retrieval. In the TRECVid 2007 automatic search task, researchers from Glasgow University investigated the transition from static keyframe matching to temporal video processing. Their study focused on the efficacy of Scale-Invariant Feature Transform (SIFT) visual features, both in isolation and when integrated into a hybrid pipeline with textual metadata and multi-frame tracking.

Methodology

The research team submitted two distinct systems to evaluate the impact of different feature sets and processing techniques.

UG_F_Sys1: Visual-Only Baseline

This system relied exclusively on SIFT features. For each video shot, a single representative keyframe was selected. SIFT points were extracted from these keyframes and compared against points from query images. Dissimilarity was calculated based on the ratio of matched points to the total number of points across both images.

UG_F_Sys2: Hybrid Visual and Textual Pipeline

The second system employed a more complex, multi-stage approach:

  1. Textual Filtering: A preliminary search was conducted using manual annotations (from TRECVid 2006 and custom labels) to identify a candidate set of videos.
  2. Multi-Keyframe Analysis: Instead of a single frame, five keyframes per shot were analyzed using SIFT.
  3. Temporal Tracking: The system tracked SIFT matches across the five frames. Shots where matches persisted over time were assigned higher weights, leveraging the temporal nature of video data.

Experimental Results

The systems were evaluated on the TRECVid 2007 test set using Mean Average Precision (MAP). The results highlighted a stark contrast between the two approaches.

Overall performance comparison between the visual-only (Sys1) and hybrid (Sys2) runs.

As shown in the table above, the visual-only run (UG_F_Sys1) achieved a MAP of only 0.001, which the authors characterized as a "disaster." In contrast, the hybrid system (UG_F_Sys2) achieved a MAP of 0.017, representing a 17-fold improvement. This disparity underscores that while SIFT is robust for identifying specific objects, it struggles with the categorical retrieval required for general video search without semantic context.

Topic-Specific Performance

Performance varied significantly across different query topics. The hybrid system excelled in scenarios with visually distinct or well-annotated subjects.

MAP scores for specific topics (217-220) and the overall average for both systems.

The system performed exceptionally well on Topic 218 ("People playing musical instruments"), reaching a MAP of 0.2639. Other relatively successful topics included Topic 220 ("Gray scale shots of streets") and Topic 206 ("Hills or mountains"). These results suggest that temporal tracking and textual anchors are particularly effective when the visual features are consistent across multiple frames.

Conclusion

The Glasgow University experiments at TRECVid 2007 demonstrate that pure visual matching using local features like SIFT is insufficient for automatic video retrieval. The significant performance gain seen in the hybrid system highlights the necessity of textual metadata as a search "anchor" and the value of temporal tracking to capture the unique characteristics of video data. Future work must focus on more advanced methods to bridge the semantic gap and reduce the dependency on manual annotations.

Summary updated

BibTeX
@inproceedings{punitha2007glasgow,
  author = {P. Punitha and Joemon M. Jose},
  title = {{Glasgow University at TRECVID 2007}},
  booktitle = {TRECVID},
  year = {2007},
  url = {https://openreview.net/forum?id=xLzBnFkohA},
}
Figure