BUPT at TRECVID 2007: Shot Boundary Detection

Zhi-Cheng Zhao, Xing Zeng, Tao Liu, An-Ni CaiTRECVID 2007 · OpenReview

Abstract

This paper details the Beijing University of Posts and Telecommunications (BUPT) team's inaugural submission to the TRECVID 2007 Shot Boundary Detection (SBD) task. The authors propose a real-time, sliding-window system that uses Support Vector Machines (SVMs) alongside specialized motion and fade detectors to identify video shot boundaries. While their hard cut detection achieved excellent precision and recall, their gradual transition detection struggled, highlighting the ongoing challenge of distinguishing true video transitions from complex camera and object motion.

Introduction

In video processing and retrieval, Shot Boundary Detection (SBD) is the foundational step of breaking a continuous video into its constituent shots. This involves detecting two main types of boundaries: abrupt changes from one shot to the next (CUTs) and slow changes like fades, dissolves, or wipes (Gradual Transitions, or GTs).

While detecting CUTs is generally well-understood, it is still prone to false positives from camera flashes or "video-in-video" editing effects. GT detection is significantly harder; the inter-frame visual differences during a gradual transition look almost identical to the differences caused by heavy camera motion (panning, zooming) or large moving objects. The BUPT system aims to build a fast, real-time system that can accurately detect both boundary types while explicitly filtering out these visual confounders.

Methodology

The BUPT system processes video in real-time using a sliding window approach, extracting a mix of visual and aural features including HSV/RGB histograms, audio MFCCs, and MPEG motion vectors. To capture temporal context, they construct "feature ties"—groups of features from all frames within the local sliding window.

The BUPT SBD system architecture, showing the pipeline from feature extraction through specialized detectors to final results.

The system relies on a pipeline of specialized detectors:

  • CUT Detector: An SVM trained on over 20,000 samples. Candidate cuts are passed through a post-processing filter based on edge-change detection to prevent false positives from flashes.
  • FOI (Fade In/Out) Detector: This detector looks for "monochrome blocks" (frames with low HSV color sums) combined with audio silence. Boundaries are then refined using an accumulated histogram method.
  • GT Detector: An SVM identifies candidate gradual transitions, which are then subjected to a rigorous motion filtering step.
  • Motion Detector: To stop camera and object motion from being misclassified as GTs, the system extracts camera motion (pan, tilt, zoom) and object motion. If motion activity exceeds specific thresholds, the candidate GT is vetoed.

Experimental Results

The system was evaluated on the official TRECVID 2007 SBD task dataset. The team submitted 10 different runs, tweaking SVM penalty parameters and motion thresholds to explore precision/recall tradeoffs.

Performance metrics for the 10 submitted runs across ALL, CUTS, and GRADUAL transition categories.

The results showed a stark contrast between CUT and GT detection performance. As seen in the performance table, CUT detection achieved highly satisfying results, with precision peaking at 0.985 (Run 4) and recall peaking at 0.966 (Run 2). Conversely, GT detection was poor; precision was generally low, ranging from 0.147 to 0.514, while recall varied wildly depending on the penalty parameters used (0.092 to 0.752).

Discussion and Future Work

The authors identified two main limitations regarding the system's performance on Gradual Transitions. First, the custom motion detector failed to work as well as expected, failing to filter out enough false positives. This suggests that a much more elaborate design for motion modeling is required. Second, inconsistencies in the TRECVID ground truth annotations—specifically regarding the exact start and end frames of transitions—led to discrepancies between the algorithm's output and the evaluation metrics. Future work will focus on refining motion filtering and improving the robustness of GT detection algorithms.

Summary updated

BibTeX
@inproceedings{zhao2007bupt,
  author = {Zhi-Cheng Zhao and Xing Zeng and Tao Liu and An-Ni Cai},
  title = {{BUPT at TRECVID 2007: Shot Boundary Detection}},
  booktitle = {TRECVID},
  year = {2007},
  url = {https://openreview.net/forum?id=s1lcgBJQls},
}
Figure