A Frame Rate Conversion Method Based on a Virtual Shutter Angle

Alex Mackin, Fan Zhang, David R. Bull

Abstract

As High Frame Rate (HFR) video becomes standard, converting these sources to legacy lower frame rates (e.g., 24 or 30 fps) remains a challenge. Traditional methods often suffer from temporal aliasing (judder) or ghosting artifacts. This paper presents a novel down-sampling framework that utilizes Motion Compensated Frame Interpolation (MCFI) to simulate a "virtual shutter angle." By synthesizing intermediate frames to fill temporal gaps and applying a Gaussian weighting scheme, the proposed method achieves smooth, filmic motion blur while significantly reducing artifacts. Subjective evaluations demonstrate a strong viewer preference for this approach, particularly at very low frame rates.

Introduction

The transition to HFR video (60 fps and above) offers superior motion clarity but creates compatibility issues with legacy display standards and cinematic preferences. Simply dropping frames to reach 24 fps results in "judder," while simple frame averaging creates "ghosting"—where moving objects appear as discrete, overlapping images. These artifacts stem from the "dark" periods in the original capture where the shutter was closed, creating gaps in the temporal information.

Proposed Methodology

The Virtual Shutter Angle

The core of the proposed method is the concept of a virtual shutter angle. Unlike traditional averaging which only uses existing frames, this approach uses MCFI to generate synthetic intermediate frames that fill the gaps between original captures.

Temporal representation of frame rate conversion showing source frames ($F_s$) and the synthesized destination frame ($F_d$) with a virtual shutter angle.

By combining original and synthesized frames, the system can simulate a continuous exposure over any desired shutter angle (up to 360°). This ensures a continuous motion path, effectively eliminating the spatial gaps that cause ghosting.

Hierarchical Prediction and Weighting

To manage computational complexity, the framework employs a hierarchical prediction structure. This allows the system to use previously predicted frames as references for higher layers, narrowing the search range for motion vectors. Furthermore, instead of a standard box filter (equal weighting), a Gaussian weighting scheme is applied to the frame sequence. This more accurately mimics the physical light integration of a camera sensor, providing a more natural-looking motion blur that balances detail preservation with aliasing suppression.

Experimental Results

The method was evaluated using 4K sequences from the Ultra Video Group (UVG) database, originally captured at 60 fps with a 180° shutter.

A visual example of the smooth, continuous motion blur achieved by the proposed method on a moving cyclist.

Subjective tests involving 18 participants using a double-stimulus methodology revealed that the proposed method significantly outperforms traditional frame averaging. At a target rate of 15 fps, over 80% of participants preferred the proposed method. Statistical analysis (one-tailed t-test) confirmed these results with high significance ($p \approx 0$).

Conclusion

By integrating motion-compensated interpolation with a virtual shutter model, this method provides a robust solution for high-quality frame rate conversion. While computationally more intensive than simple averaging, the significant gains in visual quality—particularly the elimination of ghosting—make it a compelling choice for professional video processing and archival conversion. Future work will explore the integration of deep-learning-based interpolation to further enhance performance in complex scenes.

Summary updated

Abstract

In this paper a new method for frame rate conversion is presented. Utilising motion compensated frame prediction, our method mitigates the spatial distortions associated with traditional methods, while facilitating the conversion between a wider range of frame rates - useful when converting to/between legacy video formats. Using the concept of a virtual shutter angle, our method provides content providers with greater flexibility over the motion characteristic of their video sequences. We also propose a Gaussian weighting scheme which attempts to emulate a video sequence as if it was captured natively at the converted frame rate. A subjective experiment which compared our method to averaging frames at a range of frame rates, has shown that our method results in significantly higher visual quality across a range of content - especially for high down-sample factors.
BibTeX
@inproceedings{mackin2019frame,
  author = {Alex Mackin and Fan Zhang and David R. Bull},
  title = {{A Frame Rate Conversion Method Based on a Virtual Shutter Angle}},
  booktitle = {ICIP},
  year = {2019},
  doi = {10.1109/icip.2019.8803489},
}
Figure