University of Central Florida at TRECVID 2004

Yun Zhai, Xiaochun Chao, Yunjun Zhang, Omar Javed, Alper Yilmaz, Fahd Rafi, Saad Ali, Orkun Alatas, Saad M. Khan, Mubarak ShahTRECVID 2004 · OpenReview

Introduction

The University of Central Florida (UCF) participated in the TRECVID 2004 challenge, focusing on two core tasks in video information retrieval: High-Level Feature Extraction and Story Segmentation. The goal was to bridge the "semantic gap"—the disconnect between raw pixel data and high-level human concepts like identifying specific people or activities in broadcast news.

Methodology

UCF's approach integrated visual analysis with automated speech transcripts to improve retrieval accuracy and efficiency.

Multimodal Person Spotting

To identify specific individuals, such as Bill Clinton or Madeleine Albright, the system utilized a multi-stage filtering process. Instead of processing every frame, the system first scanned speech-to-text "wordtime" data. If a target name was mentioned, the system applied a Haar-like face detector to the relevant video segment. Identity was then confirmed using Eigenfaces and Support Vector Machines (SVM).

Multimodal Person Spotting Pipeline

Feature Extraction with Adaboost

For broader features like "Beach" or "Basketball," the team used the Adaboost algorithm. This method selected discriminative low-level features, including color histograms and correlograms. To improve generalization, the training set was augmented with images manually collected from the web.

Story Segmentation

A key innovation in story segmentation was the use of "body regions" to identify news anchors. Recognizing that face detection can be unreliable at low resolutions, the team focused on the torso/clothing area below the face. Since anchors typically wear the same attire throughout a broadcast, color histograms of these body regions served as a stable "fingerprint" for clustering shots. Additionally, a Shot Connectivity Graph (SCG) was used to detect recurring news desk scenes, marking story boundaries.

Results

The UCF team achieved notable success in several categories, particularly in visual feature detection.

  • Beach Detection: Ranked 1st out of 59 runs with an average precision of 0.0635.
  • Basketball Scored: Achieved an average precision of 0.2632, ranking 25th out of 63.
  • Story Segmentation: Achieved a precision of 0.5390 and a high recall of 0.8030.
Performance of the Beach Feature Detector

Discussion and Key Insights

The 2004 entry highlighted several practical lessons in multimodal retrieval:

  • Text as a Filter: Using speech transcripts as a computational filter significantly reduced the search space for expensive face detection tasks.
  • Labeling Nuance: The team found that searching for specific titles (e.g., "President") was more effective than first names (e.g., "Bill") for identifying public figures in news contexts.
  • Spatial Heuristics: Incorporating spatial logic (e.g., sky in the upper frame, sand in the lower) was vital for the top-ranked beach detector.

Despite these successes, limitations remained. The face detection system struggled with non-frontal views and crowded scenes. Furthermore, relying on a single keyframe per shot was often insufficient for robust recognition, suggesting that future systems should leverage temporal evidence across multiple frames.

Summary updated

BibTeX
@inproceedings{zhai2004university,
  author = {Yun Zhai and Xiaochun Chao and Yunjun Zhang and Omar Javed and Alper Yilmaz and Fahd Rafi and Saad Ali and Orkun Alatas and Saad M. Khan and Mubarak Shah},
  title = {{University of Central Florida at TRECVID 2004}},
  booktitle = {TRECVID},
  year = {2004},
  url = {https://openreview.net/forum?id=HMtKuFMVZO},
}
Figure