Movie summarization based on audiovisual saliency detection

Georgios Evangelopoulos, Konstantinos Rapantzikos, Alexandros Potamianos, Petros Maragos, Nancy Zlatintsi, Yannis Avrithis

Abstract

Based on perceptual and computational attention modeling studies, we formulate measures of saliency for an audiovisual stream. Audio saliency is captured by signal modulations and related multi-frequency band features, extracted through nonlinear operators and energy tracking. Visual saliency is measured by means of a spatiotemporal attention model driven by various feature cues (intensity, color, motion). Audio and video curves are integrated in a single attention curve, where events may be enhanced, suppressed or vanished. The presence of salient events is signified on this audiovisual curve by geometrical features such as local extrema, sharp transition points and level sets. An audiovisual saliency-based movie summarization algorithm is proposed and evaluated. The algorithm is shown to perform very well in terms of summary informativeness and enjoyability for movie clips of various genres.
BibTeX
@inproceedings{evangelopoulos2008movie,
  author = {Georgios Evangelopoulos and Konstantinos Rapantzikos and Alexandros Potamianos and Petros Maragos and Nancy Zlatintsi and Yannis Avrithis},
  title = {{Movie summarization based on audiovisual saliency detection}},
  booktitle = {ICIP},
  year = {2008},
  doi = {10.1109/icip.2008.4712308},
}

Loading related content...

Figure