Abstract
Standard video frame interpolation (VFI) algorithms typically assume continuous motion, relying on optical flow or trajectory prediction. However, modern video content frequently includes "discontinuous motions"—digital overlays such as subtitles, logos, and UI elements—that do not follow physical laws. These elements often cause standard VFI models to produce ghosting or blurring artifacts. We propose a plug-and-play framework that explicitly identifies and handles these discontinuities. By utilizing Explicit Coherence Guidance (ECG) and a dedicated D-map estimator, our method preserves the sharpness of digital overlays without requiring retraining of the underlying VFI model.
Introduction
Most VFI algorithms are built on the assumption of continuous motion, using trajectories to predict pixel movement. This works well for natural scenes but fails on discontinuous motion: static logos, sudden subtitles, or game menus. Standard models attempt to "warp" these elements, resulting in "fading" or "blending" artifacts. While previous attempts used synthetic data augmentation, they often failed to generalize to real-world complexities.
Our approach defines natural motion by its spatio-temporal coherence and perpetual nature. Anything failing these criteria is classified as discontinuous. This "definition by negation" allows for a robust identification of digital artifacts.
Methodology
The core of our framework is a supervisor system for existing VFI models, consisting of several key components:
Explicit Coherence Guidance (ECG)
The central innovation is the use of a frozen VFI model to interpolate frames at different time steps. Continuous motion is perpetual and changes over time, whereas static or discontinuous elements often remain identical across interpolated results. By comparing these outputs, we generate a "coherence mask" ($M_c$) that guides the identification of discontinuous pixels.
D-map Estimator and Copying Strategy
A small, trainable network produces a binary "Discontinuity map" (D-map). For pixels flagged as discontinuous, the system employs a Copying Strategy, simply taking the pixel from the adjacent original frame ($I_1$) instead of interpolating. For continuous pixels, the standard VFI output ($I_c$) is used. This hybrid approach ensures that natural motion remains smooth while digital overlays stay sharp.
Supervision and Augmentation
We use the error map of a pre-trained VFI model as a supervisory signal ($L_e$), as failure points often coincide with discontinuities. Additionally, we use FTM+ Augmentation, expanding synthetic data to include complex polygons, arcs, and alpha-blended transparency to better simulate real-world UI.
Experiments and Results
We evaluated our framework using standard benchmarks like Vimeo-90K, UCF101, and DAVIS, alongside the GDM (Graphical Discontinuous Motions) dataset, which features screen recordings rich in UI elements and text.
Quantitative and Qualitative Performance
On the GDM dataset, our framework achieved significant performance gains, improving PSNR by 2.9 to 3.7 dB across various base models (e.g., AdaCoF, CAIN, EMA-VFI). Qualitatively, the framework maintains sharp, legible outlines for text and logos where other models produce warped or ghostly artifacts.
Conclusion
By keeping the underlying VFI model frozen, our framework avoids "catastrophic forgetting" and remains highly adaptable. While it introduces some computational overhead due to multiple VFI passes during guidance, it provides a pragmatic and effective solution for real-world video interpolation challenges. Future work will explore more sophisticated copying logic for non-physical but moving digital elements.
Summary updated