Introduction
The University of Central Florida (UCF) participated in the TRECVID 2004 challenge, focusing on two core tasks in video information retrieval: High-Level Feature Extraction and Story Segmentation. The goal was to bridge the "semantic gap"—the disconnect between raw pixel data and high-level human concepts like identifying specific people or activities in broadcast news.
Methodology
UCF's approach integrated visual analysis with automated speech transcripts to improve retrieval accuracy and efficiency.
Multimodal Person Spotting
To identify specific individuals, such as Bill Clinton or Madeleine Albright, the system utilized a multi-stage filtering process. Instead of processing every frame, the system first scanned speech-to-text "wordtime" data. If a target name was mentioned, the system applied a Haar-like face detector to the relevant video segment. Identity was then confirmed using Eigenfaces and Support Vector Machines (SVM).
Feature Extraction with Adaboost
For broader features like "Beach" or "Basketball," the team used the Adaboost algorithm. This method selected discriminative low-level features, including color histograms and correlograms. To improve generalization, the training set was augmented with images manually collected from the web.
Story Segmentation
A key innovation in story segmentation was the use of "body regions" to identify news anchors. Recognizing that face detection can be unreliable at low resolutions, the team focused on the torso/clothing area below the face. Since anchors typically wear the same attire throughout a broadcast, color histograms of these body regions served as a stable "fingerprint" for clustering shots. Additionally, a Shot Connectivity Graph (SCG) was used to detect recurring news desk scenes, marking story boundaries.
Results
The UCF team achieved notable success in several categories, particularly in visual feature detection.
- Beach Detection: Ranked 1st out of 59 runs with an average precision of 0.0635.
- Basketball Scored: Achieved an average precision of 0.2632, ranking 25th out of 63.
- Story Segmentation: Achieved a precision of 0.5390 and a high recall of 0.8030.
Discussion and Key Insights
The 2004 entry highlighted several practical lessons in multimodal retrieval:
- Text as a Filter: Using speech transcripts as a computational filter significantly reduced the search space for expensive face detection tasks.
- Labeling Nuance: The team found that searching for specific titles (e.g., "President") was more effective than first names (e.g., "Bill") for identifying public figures in news contexts.
- Spatial Heuristics: Incorporating spatial logic (e.g., sky in the upper frame, sand in the lower) was vital for the top-ranked beach detector.
Despite these successes, limitations remained. The face detection system struggled with non-frontal views and crowded scenes. Furthermore, relying on a single keyframe per shot was often insufficient for robust recognition, suggesting that future systems should leverage temporal evidence across multiple frames.
Summary updated