Abstract Automatic movie trailer generation requires a shift from traditional video summarization, which focuses on narrative continuity, to a model centered on visual attractiveness. This paper proposes a novel framework that quantifies attractiveness using eye-tracking data, specifically fixation variance. By modeling the decay of viewer attention within a shot as a self-correcting point process and aligning cuts with musical saliency through graph-based optimization, the system generates trailers that rival professional productions in engagement and rhythm.
Introduction
Creating a movie trailer is a specialized art form. Unlike standard video summarization, which aims to provide a concise "gist" of a story, a trailer is designed to maximize emotional impact and viewer engagement—a quality the authors define as "attractiveness." Traditionally, this process requires human editors to select high-impact shots and arrange them rhythmically. Automating this task is difficult because attractiveness is subjective and hard to quantify. Furthermore, a trailer is not just a collection of good shots; it is a rhythmic sequence where the timing of cuts is crucial to maintaining viewer interest.
Quantifying Visual Attractiveness
The researchers propose that "attractiveness" can be objectively measured through viewer attention. Using eye-tracking data, they discovered a strong correlation between shot content and fixation variance ($\sigma$). In highly engaging shots, such as fast-paced action scenes or close-ups of main characters, viewers tend to focus on the same focal points, resulting in low fixation variance. Conversely, in less engaging shots like landscapes or credit sequences, viewer attention wanders, leading to high fixation variance.
Modeling Attention Dynamics
A key insight of this work is that attractiveness is not static; it decays over the duration of a shot. The authors model this "loss of attractiveness" using a self-correcting point process. As shown in Figure 1(e), fixation variance typically follows an exponential growth curve within a single shot, indicating that viewers lose interest the longer a shot lingers.
In this framework, a "montage" (a cut to a new shot) acts as a corrective event. A well-timed transition resets viewer attention, spiking interest and lowering fixation variance once again. This mathematical approach justifies the fast-paced nature of modern trailers as a tool for psychological engagement.
Trailer Assembly and Optimization
To generate the final trailer, the system must synchronize these visual "resets" with background music. The process involves three steps:
- Musical Saliency Detection: Using Discrete Cosine Transform (DCT), the algorithm identifies rhythmic peaks in the music.
- Graph Construction: The system builds a "trellis graph" where nodes represent candidate video shots and edges represent potential transitions.
- Shortest Path Optimization: The algorithm finds the optimal sequence of shots that minimizes the total "loss of attractiveness" while ensuring that cuts align precisely with the music's saliency points. This ensures a high-energy, rhythmic flow similar to professional editing.
Results and Evaluation
The model was tested on a dataset of 16 movies from 2012 and 2014. The performance was evaluated against commercial summarization software (Muvee), previous research models (V2T), and professional human-made trailers (RT).
- Objective Engagement: Eye-tracking tests on the generated trailers showed that the proposed method achieved significantly lower fixation variance than other automated tools, closely approaching the performance of professional trailers.
- Subjective Quality: Human volunteers rated the trailers on Rhythm, Attractiveness, and Appropriateness. The proposed model consistently outperformed baselines, particularly in rhythmic synchronization.
- Efficiency: The algorithm is computationally efficient, operating with $O(N)$ complexity for training and $O(N^2)$ for testing, making it suitable for large-scale video processing.
Conclusion
By turning the "fuzzy" concept of attractiveness into a measurable signal (fixation variance) and modeling the temporal dynamics of attention, this research provides a robust framework for automated creative editing. While current limitations include a lack of semantic and narrative understanding (e.g., plot points or dialogue), the point process-based model represents a significant step forward in bridging the gap between machine-generated summaries and human-edited cinematic experiences.
Summary updated