Non-Linear Motion Modeling for Video Frame Interpolation

This direction is split into other directions: linear motion assumption, non-linear motion modeling (linear motion assumption). Its last version stays readable and citable here.

Version 1 of 1267 papers302 conceptsfrozen 2026-10-04

Most video frame interpolation (VFI) algorithms operate on a simple, intuitive assumption: between two captured frames, objects move in a straight line at a constant speed. This is the linear motion assumption. While this approximation works for high-frame-rate footage where the "gap" between frames is tiny, it falls apart in the real world. A bouncing ball, a spinning umbrella, or a car slamming on its brakes all violate this rule.

When a model assumes linear motion but the reality is curved or accelerating, the result is "ghosting"—visual artifacts where the object appears in two places at once or becomes a blurry mess. To solve this, researchers are moving toward models that understand the physics of acceleration and complex trajectories.

The Failure of Constant Velocity

In standard interpolation, such as the early work in Super SloMo by Huaizu Jiang and colleagues, the goal is to hallucinate intermediate frames. If an object is at position A at time 0 and position B at time 1, a linear model places it exactly halfway between A and B at time 0.5.

However, as noted in Compensating for Motion Estimation Inaccuracies in DVC, real-world motion is rarely so polite. Objects accelerate due to gravity, change direction abruptly, or follow curved paths. Amit Agrawal and Ramesh Raskar observed as early as 2007 that even motion blur—the "enemy" of image quality—is actually a record of these non-linear paths, and treating it as a simple linear streak leads to poor reconstruction.

From Linear to Quadratic Models

The first major leap beyond the linear trap was the introduction of Quadratic Video Interpolation (QVI). Instead of looking at just two frames to estimate a straight path, Xiangyu Xu and Wenxiu Sun proposed using four consecutive frames.

By analyzing a sequence, the model can calculate the change in velocity—acceleration. This allows the model to fit a parabola rather than a line to the object's path. This approach was further refined in Enhanced Quadratic Video Interpolation (EQVI), which won the AIM 2020 challenge by using least-squares flow rectification to handle even more complex dynamics. More recently, IQ-VFI has shown that it is possible to "distill" this quadratic understanding into a model that only needs two frames, learning to infer latent acceleration from the appearance of the pixels themselves.

Seeing in the Dark: Event-Based Interpolation

Even with quadratic math, some motions are too fast or erratic to be captured by standard cameras. When a balloon pops or a camera shakes violently, the "blind time" between frames contains too much information to guess.

Researchers like Daniel Gehrig and Stepan Tulyakov introduced Time Lens, which uses event cameras. Unlike traditional sensors that take "snapshots," event cameras record a continuous stream of brightness changes. This provides a high-temporal-resolution map of exactly how an object moved between frames. By combining these "events" with RGB frames, models like TimeLens-XL and A²OF can reconstruct highly non-linear, high-speed movements that were previously impossible to interpolate.

Beyond Rigid Geometry

The most recent frontier moves away from fixed mathematical formulas (like linear or quadratic) entirely. Xiaohong Liu and colleagues proposed Generalized Deformable Convolution (GDConv), which allows the neural network to sample pixels from anywhere in space and time. Instead of being forced into a specific trajectory, the model learns the "shape" of the motion directly from the data.

This flexibility is crucial for "unnatural" motion, such as the sudden appearance of subtitles or UI overlays in video games, which Beyond Natural Motion identifies as a major failure point for physics-based models. By treating motion as a learnable, non-linear field rather than a simple displacement vector, these models are finally beginning to bridge the gap between the rigid math of computer vision and the fluid, unpredictable motion of the real world.

Papers in this version

Showing 60 of 267 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_non_linear_motion_modeling_for_video_fra_32211bee,
  title = {Non-Linear Motion Modeling for Video Frame Interpolation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-linear-motion-modeling-for-video-frame-interpolation/rdr_32211bee185f49f1b4d0cb8b9bd7b2c4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_non_linear_motion_modeling_for_video_fra_32211bee_v1,
  title = {Non-Linear Motion Modeling for Video Frame Interpolation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-linear-motion-modeling-for-video-frame-interpolation/rdr_32211bee185f49f1b4d0cb8b9bd7b2c4/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}