Broadcast News Parsing using Visual Cues: A Robust Face Detection Approach

Yannis Avrithis, Nicolas Tsapatsoulis, Stefanos D. KolliasIEEE International Conference on Multimedia and Expo (III) 2000 · DOI

TL;DR

This paper introduces a robust method for automatically parsing and indexing broadcast news videos using only visual cues, specifically focusing on face detection. This approach is valuable for creating interactive news services and can function independently or complement existing audio-visual analysis systems, especially for archives lacking textual transcripts.

What Problem Does This Paper Address?

The explosion of multimedia data, particularly from television broadcasts and digital archives, necessitates efficient tools for content analysis, parsing, and indexing. While existing methods for tasks like news summarization and retrieval often rely heavily on textual cues (closed captions, teletext), a significant portion of broadcast content, especially older archives, lacks this information. This creates a need for robust methods that can leverage visual information alone to segment news into meaningful units and identify key elements like anchorpersons and reports. The challenge lies in accurately identifying these elements from raw video streams, which are inherently unstructured.

Methods

The core of the proposed technique is a multi-stage approach to robust face detection, followed by shot classification.

  • Face Detection:
    • Color Segmentation: The paper utilizes the Multiresolution Recursive Shortest Spanning Tree (M-RSST) algorithm for initial image segmentation based on color.
  • Skin-Tone Color Matching: This step refines the segmentation by leveraging the fact that skin tones occupy a specific region in the YCrCb color space. A Gaussian density function is used to model skin-tone color distribution, and segments are merged based on their skin-color probability.
  • Shape Processing: To further refine face detection and discard non-face objects with similar colors, global shape features like compactness (ratio of area to perimeter squared) and elongation (derived from the covariance matrix of contour points) are extracted. These features are normalized and invariant to translation, scaling, and rotation.
  • Face Probability: Finally, a weighted combination of skin-color probability and shape features is used to compute an overall face probability for each segment.
  • News Shot Classification:
    • Once face detection is performed, the system identifies "face close-ups" which are indicative of anchorpersons or reporters.
  • Segments are filtered based on size (discarding very small faces) and resolution (less than 32x32 pixels).
  • Fuzzy membership functions are used to classify frames based on dominant face location and size.
  • Temporal fluctuations of these properties, along with background motion estimation (using absolute frame differences), are used to classify shots into categories: single/double anchor (face close-up, static background), reports/interviews (face close-up, moving background), static images (fixed background, no dominant faces), and outdoor shots (significant motion, small/no faces).

Datasets

The experiments were conducted on a database of Greek TV news recordings from channels A5, ET1, MEGA, and ANT1. Six news broadcasts, each 10 minutes long, were recorded at 10 frames per second with a resolution of 384x288 pixels. The paper does not mention the creation of new benchmarks but rather uses these recordings for evaluation.

Key Insights and Ideas

  • Visual Cues Alone are Powerful: The paper demonstrates that robust face detection and subsequent shot classification can be achieved using only visual information, making it applicable to a wider range of video content, including historical archives.
  • Multi-stage Refinement: The combination of color segmentation, skin-tone matching, and shape analysis provides a powerful and robust mechanism for accurately detecting faces, even in challenging conditions.
  • Shape Features for Disambiguation: Using global shape features (compactness, elongation) is crucial for filtering out non-face objects that might coincidentally match skin-color models, thus reducing false alarms.
  • Temporal Context is Key for Classification: While face detection is frame-based, classifying entire shots requires considering temporal aspects like background motion and the duration of face presence.

Key Results

The paper reports promising results for shot classification across six test sequences.

  • Anchorperson shots achieved the highest classification rates, with precision and recall values often exceeding 0.90.
  • Report/interview shots showed slightly lower rates (around 0.80-0.90) due to uncontrolled illumination conditions that can interfere with the skin-color model.
  • Static shots had limited data, making it difficult to draw firm conclusions.
  • The system demonstrated the ability to correctly identify dominant face segments and filter out false alarms, as illustrated in Figure 3. For instance, a false alarm caused by a segment with a face-like shape fitting the skin-color model was alleviated by considering dominant face motion.

Limitations and Future Work

  • Limitations:
    • The classification rates for report/interview shots are impacted by uncontrolled illumination, leading to misclassifications with outdoor shots.
  • The limited number of static shots prevented drawing reliable conclusions about their classification.
  • The paper acknowledges that for higher-level semantic segmentation into true news topics, audio information (like speaker identification) and closed caption transcripts would be necessary.
  • Future Work:
    • The authors suggest that their robust face detection procedure can be integrated with audio and textual cues from existing systems to achieve more comprehensive semantic segmentation.
  • Further work could explore more sophisticated methods for shot classification and semantic understanding of news content.

Summary updated

BibTeX
@inproceedings{avrithis2000broadcast,
  author = {Yannis Avrithis and Nicolas Tsapatsoulis and Stefanos D. Kollias},
  title = {{Broadcast News Parsing using Visual Cues: A Robust Face Detection Approach}},
  booktitle = {IEEE International Conference on Multimedia and Expo (III)},
  year = {2000},
  doi = {10.1109/icme.2000.871044},
}
Figure