Universität Karlsruhe (TH) at TRECVID 2008

Hazim Kemal Ekenel, Hua Gao, Rainer Stiefelhagen

Abstract

This paper details the refinements made by the Universität Karlsruhe (TH) team to their video concept detection system for the TRECVID 2008 evaluation. By addressing critical bottlenecks such as extreme class imbalance and rigid feature fusion, the team significantly improved performance. The revised system achieved a mean inferred Average Precision (infAP) of 6.3% on 2007 benchmarks, a substantial increase from the 2.3% baseline.

Introduction

The "semantic gap" remains a primary challenge in video retrieval, representing the disconnect between low-level pixel data and high-level human concepts like "Airplane flying" or "Cityscape." Following a below-average performance in the TRECVID 2007 competition, researchers at Universität Karlsruhe (TH) identified three key areas for improvement: inefficient feature representation, severe class imbalance in training data, and the lack of flexibility in fusing disparate visual cues.

Methodology

The team transitioned from a general-purpose framework to a highly optimized pipeline designed to handle the nuances of video data.

Feature Extraction and Classification

The system utilizes a suite of visual descriptors, including color histograms, 5x5 grid color moments in Lab color space, color correlograms, and edge histograms. For classification, the team employed Support Vector Machines (SVMs). A critical optimization involved swapping the standard Gaussian RBF kernel for a Laplacian RBF kernel using L1 distance for histogram-based features, as L1 metrics often provide superior performance for discrete probability densities.

Addressing Class Imbalance

Video datasets are characterized by extreme sparsity of positive examples. To prevent SVMs from becoming biased toward the majority negative class, the researchers implemented a down-sampling strategy. While all positive samples were retained, the number of negative samples was limited to create a more balanced training environment, forcing the model to better distinguish the features of rare concepts.

Category-Specific Feature Fusion

Recognizing that different visual features vary in utility depending on the concept (e.g., color is highly relevant for "Flower" but less so for "Telephone"), the team moved away from global weighting. Instead, they implemented a weighted sum fusion rule where weights were assigned to each feature on a per-category basis, determined by validation performance.

Evaluation and Results

The system was evaluated using standard NIST benchmarks, including TRECVID 2007 and 2008 datasets.

Comparison of system performance across different evaluation cycles.

As shown in the table above, the revised system demonstrated a marked improvement, nearly tripling the baseline performance on the 2007 test data. In the official TRECVID 2008 run, the best configuration (UKA2) achieved a mean infAP of 3.8%.

Detailed infAP results across 20 evaluated concepts for the UKA1 and UKA2 runs.

Performance varied significantly across different semantic categories. The system excelled in identifying "Nighttime" (0.130 infAP) and "Boat/Ship" (0.113 infAP). However, it struggled with highly specific or extremely rare concepts, such as "Emergency Vehicle," where it achieved an infAP of 0.000. While these results were competitive with the median performance of TRECVID participants, they trailed the top-performing systems that utilized more extensive training data and complex temporal modeling.

Discussion and Future Work

The improvements in the 2008 system highlight the importance of addressing data imbalance and employing flexible fusion strategies. However, limitations remain. The current system relies on global key-frame features and does not yet leverage localized object-level annotations (bounding boxes), which became a focus in the 2008 evaluation. Furthermore, the researchers noted that their system is "data-hungry"; future work will focus on incorporating larger training sets from previous TRECVID cycles and exploring more sophisticated spatial-temporal modeling to further bridge the semantic gap.

Summary updated

BibTeX
@inproceedings{ekenel2008universitat,
  author = {Hazim Kemal Ekenel and Hua Gao and Rainer Stiefelhagen},
  title = {{Universität Karlsruhe (TH) at TRECVID 2008}},
  booktitle = {TRECVID},
  year = {2008},
  url = {https://openreview.net/forum?id=OEdmG1jHFR9G},
}
Figure