Trailer Generation via a Point Process-Based Visual Attractiveness Model

Hongteng Xu, Yi Zhen, Hongyuan Zha

Abstract Automatic movie trailer generation requires a shift from traditional video summarization, which focuses on narrative continuity, to a model centered on visual attractiveness. This paper proposes a novel framework that quantifies attractiveness using eye-tracking data, specifically fixation variance. By modeling the decay of viewer attention within a shot as a self-correcting point process and aligning cuts with musical saliency through graph-based optimization, the system generates trailers that rival professional productions in engagement and rhythm.

Introduction

Creating a movie trailer is a specialized art form. Unlike standard video summarization, which aims to provide a concise "gist" of a story, a trailer is designed to maximize emotional impact and viewer engagement—a quality the authors define as "attractiveness." Traditionally, this process requires human editors to select high-impact shots and arrange them rhythmically. Automating this task is difficult because attractiveness is subjective and hard to quantify. Furthermore, a trailer is not just a collection of good shots; it is a rhythmic sequence where the timing of cuts is crucial to maintaining viewer interest.

Quantifying Visual Attractiveness

The researchers propose that "attractiveness" can be objectively measured through viewer attention. Using eye-tracking data, they discovered a strong correlation between shot content and fixation variance ($\sigma$). In highly engaging shots, such as fast-paced action scenes or close-ups of main characters, viewers tend to focus on the same focal points, resulting in low fixation variance. Conversely, in less engaging shots like landscapes or credit sequences, viewer attention wanders, leading to high fixation variance.

Figure 1: Examples of fixation variance across different shot types (a-d) and the dynamics of attention decay within a single shot (e).

Modeling Attention Dynamics

A key insight of this work is that attractiveness is not static; it decays over the duration of a shot. The authors model this "loss of attractiveness" using a self-correcting point process. As shown in Figure 1(e), fixation variance typically follows an exponential growth curve within a single shot, indicating that viewers lose interest the longer a shot lingers.

In this framework, a "montage" (a cut to a new shot) acts as a corrective event. A well-timed transition resets viewer attention, spiking interest and lowering fixation variance once again. This mathematical approach justifies the fast-paced nature of modern trailers as a tool for psychological engagement.

Trailer Assembly and Optimization

To generate the final trailer, the system must synchronize these visual "resets" with background music. The process involves three steps:

  1. Musical Saliency Detection: Using Discrete Cosine Transform (DCT), the algorithm identifies rhythmic peaks in the music.
  2. Graph Construction: The system builds a "trellis graph" where nodes represent candidate video shots and edges represent potential transitions.
  3. Shortest Path Optimization: The algorithm finds the optimal sequence of shots that minimizes the total "loss of attractiveness" while ensuring that cuts align precisely with the music's saliency points. This ensures a high-energy, rhythmic flow similar to professional editing.

Results and Evaluation

The model was tested on a dataset of 16 movies from 2012 and 2014. The performance was evaluated against commercial summarization software (Muvee), previous research models (V2T), and professional human-made trailers (RT).

  • Objective Engagement: Eye-tracking tests on the generated trailers showed that the proposed method achieved significantly lower fixation variance than other automated tools, closely approaching the performance of professional trailers.
  • Subjective Quality: Human volunteers rated the trailers on Rhythm, Attractiveness, and Appropriateness. The proposed model consistently outperformed baselines, particularly in rhythmic synchronization.
  • Efficiency: The algorithm is computationally efficient, operating with $O(N)$ complexity for training and $O(N^2)$ for testing, making it suitable for large-scale video processing.

Conclusion

By turning the "fuzzy" concept of attractiveness into a measurable signal (fixation variance) and modeling the temporal dynamics of attention, this research provides a robust framework for automated creative editing. While current limitations include a lack of semantic and narrative understanding (e.g., plot points or dialogue), the point process-based model represents a significant step forward in bridging the gap between machine-generated summaries and human-edited cinematic experiences.

Summary updated

Abstract

Producing attractive trailers for videos needs human expertise and creativity, and hence is challenging and costly. Different from video summarization that focuses on capturing storylines or important scenes, trailer generation aims at producing trailers that are attractive so that viewers will be eager to watch the original video. In this work, we study the problem of automatic trailer generation, in which an attractive trailer is produced given a video and a piece of music. We propose a surrogate measure of video attractiveness named fixation variance, and learn a novel self-correcting point process-based attractiveness model that can effectively describe the dynamics of attractiveness of a video. Furthermore, based on the attractiveness model learned from existing training trailers, we propose an efficient graph-based trailer generation algorithm to produce a max-attractiveness trailer. Experiments demonstrate that our approach outperforms the state-of-the-art trailer generators in terms of both quality and efficiency.
BibTeX
@inproceedings{xu2015trailer,
  author = {Hongteng Xu and Yi Zhen and Hongyuan Zha},
  title = {{Trailer Generation via a Point Process-Based Visual Attractiveness Model}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2015},
}
Figure