Glasgow University at TRECVID 2009

P. Punitha, Hemant Misra, Reede Ren, David Hannah, Anuj Goyal, Robert Villa, Joemon M. JoseTRECVID 2009 · OpenReview

Abstract

This paper details the University of Glasgow’s participation in the TRECVID 2009 automatic search task. The research focuses on bridging the "semantic gap"—the disparity between low-level visual features and high-level human concepts—by employing adaptive feature selection, diversity-based re-ranking, and Latent Dirichlet Allocation (LDA) topic modeling. Our results demonstrate that while semantic fusion significantly boosts top-tier precision, LDA-based modeling of visual "words" offers a more robust framework for general retrieval across varied search topics.

Introduction: Addressing the Semantic Gap

In 2009, the primary challenge in video retrieval remained the "semantic gap." Most automated systems struggled to map low-level pixel data, such as color and texture, to complex human queries like "a person singing." Furthermore, standard retrieval often produced redundant results, where the top-ranked items were visually near-identical, limiting the utility for users seeking diverse examples. The University of Glasgow’s entry addressed these issues by investigating automated feature selection and methods to ensure both relevance and visual diversity in search outputs.

Methodology

The team submitted six experimental runs utilizing four primary strategies:

  1. Adaptive Feature Selection: Rather than using static weights, the system dynamically identified features (e.g., color, texture) that preserved the most diversity among query examples. Statistical t-tests and correlation matrices were used to select features with high variance, which were hypothesized to be more effective at retrieving distinct, relevant results.
  2. Cluster-Based Re-ranking: To mitigate redundancy, a single-linkage clustering method based on Kruskal’s algorithm was applied. Results were re-ranked by selecting one image from each cluster in a round-robin fashion, effectively promoting visual variety to the top of the results list.
  3. LDA-based Retrieval: Scale-Invariant Feature Transform (SIFT) points were treated as a "bag-of-visual-words." By applying Latent Dirichlet Allocation (LDA), images were modeled as distributions of latent topics. Similarity between queries and images was then calculated using KL-divergence within this topic space.
  4. High-Level Feature (HLF) Fusion: Semantic concepts (e.g., "ships," "boats") provided by external teams were integrated through manual query expansion, mapping search topics to pre-defined semantic categories.

Datasets

The benchmark for these experiments was the TRECVID 2009 automatic search task. The TRECVID 2008 Development Set was utilized for model training. Additionally, the authors incorporated external High-Level Feature extractions provided by NIST and top-performing teams from the previous year (including IBM and Marburg) to supplement their visual-only runs.

Key Insights: Variance and Visual Words

A central finding of this research is that feature variance can serve as a proxy for relevance; features that are statistically distinct across query examples are often the most useful for retrieval. Furthermore, the success of the LDA approach confirms that Natural Language Processing techniques can be effectively adapted for computer vision. By clustering SIFT points into a vocabulary of 10,000 "visual words," the system successfully discovered latent visual themes that traditional global features might overlook. Finally, the authors emphasize that for end-users, diversity is often as important as precision, justifying the use of cluster-based re-ranking to provide a broader overview of a concept.

Results and Performance Analysis

The performance of the submitted runs varied significantly depending on the metric and the specific search topic.

Summary of performance metrics (MAP, P(10), R-prec, infAP, and Recall) across the six submitted runs.

As shown in Figure 5, the LDA-based run (UG-HERun6_6) achieved the highest Mean Average Precision (MAP) of 0.0132, suggesting that the latent topic approach is more robust across a wide range of queries. However, the run combining visual features with high-level semantic features (UG-PURun1_1) performed best in terms of Precision at 10 (P@10), reaching 0.1542. This indicates that semantic labels are particularly effective at ensuring the accuracy of the very top results.

Comparison of the best performing Glasgow University run against the overall TRECVID 2009 best for each search topic.

Topic-specific analysis revealed that the LDA approach was exceptionally strong for visual queries such as Topic #285 ("printed, typed, or handwritten text"), as seen in the peak in Figure 3. Conversely, hybrid visual/semantic runs excelled at topics like #281 ("people singing/playing instruments"). Despite these successes, Figure 3 also highlights a substantial performance gap between the UG runs and the state-of-the-art "Best in TV09" for many topics, underscoring the difficulty of the task.

Conclusion and Future Work

The experiments highlighted both the strengths and limitations of the current framework. While the "Bag of Visual Words" approach with LDA showed promise, simpler global feature models (e.g., UG-RRRun4_4) struggled to capture hidden concepts. Future work will focus on extending the bag-of-words framework to include high-level concepts directly and exploring hierarchical document modeling to better bridge the gap between local visual details and global semantic scope.

Summary updated

BibTeX
@inproceedings{punitha2009glasgow,
  author = {P. Punitha and Hemant Misra and Reede Ren and David Hannah and Anuj Goyal and Robert Villa and Joemon M. Jose},
  title = {{Glasgow University at TRECVID 2009}},
  booktitle = {TRECVID},
  year = {2009},
  url = {https://openreview.net/forum?id=VY9KIutZDT},
}
Figure