When capturing fast motion in low light, cameras face a fundamental physical trade-off. To gather enough light for a clear image, the shutter must stay open longer, but a long exposure causes moving objects to "smear" across the sensor, resulting in motion blur. Conversely, high-speed video requires short exposures and high frame rates, which often lead to dark, noisy, or choppy results.
Traditionally, computer vision has treated these as two distinct problems: deblurring (restoring spatial sharpness) and video frame interpolation (VFI, increasing temporal resolution). However, a growing body of research suggests that treating these tasks as a single, unified process is not just more efficient—it is mathematically more principled.
The Failure of Cascaded Pipelines
The most intuitive way to handle a blurry, low-frame-rate video is a "cascade": first deblur the existing frames, then interpolate new ones between them. But this approach suffers from severe error propagation.
As noted by Shen Wang and Wang Shen in their work on BIN (Blurry video frame INterpolation), sequential pipelines often lead to jittery results. If the initial deblurring step is imperfect, the interpolation step will amplify those artifacts, creating "ghosting" or unnatural motion. Furthermore, Meiguang Jin and Paolo Favaro argue that sequential approaches discard essential temporal information. By deblurring first, a model might strip away the very motion cues—the "smear"—that indicate exactly where an object was during the exposure.
Blur as a Temporal Signal
The core intuition behind unified frameworks is that motion blur is not just noise; it is a temporal integration of the scene. A blurry pixel is essentially the sum of all light that hit that sensor location during the exposure window.
Instead of fighting the blur, researchers like Jihyong Oh and Munchurl Kim treat it as a feature. Their framework, DeMFI (Deep Joint Deblurring and Multi-Frame Interpolation), uses the accumulated light within blurry frames to guide the reconstruction of intermediate frames. By viewing the problem as a single joint task, the model can use the "temporal smear" to more accurately estimate object trajectories.
This shift in perspective transforms the problem from "cleaning up an image" to "reconstructing a continuous latent video." As Xiang Zhang and Lei Yu demonstrate in EVDI++ (Event-based Video Deblurring and Interpolation), the goal is to estimate the latent sharp image at any arbitrary timestamp, whether that timestamp falls during an exposure or in the gap between frames.
Anchoring Reconstruction with Event Cameras
A major breakthrough in this field involves the use of event cameras—sensors that record changes in brightness at microsecond resolution rather than capturing full frames at fixed intervals.
In a unified framework, event data acts as a "continuous anchor." Christos Sakaridis and Jingyun Liang developed REFID (Event-Based Frame Interpolation with Ad-hoc Deblurring) to leverage this. Because event cameras provide a high-frequency record of motion between and during frames, they provide the missing link that allows a model to "unroll" a blurry frame into a sequence of sharp ones.
This is often formulated as a Learnable Double Integral (LDI). By integrating the stream of events, the system can mathematically relate the blurry frame (the integral of light) to the sharp frames (the instantaneous light). This unified mathematical formulation, explored by Xiang Zhang in the EVDI (Unifying Motion Deblurring and Frame Interpolation with Events) series, allows for self-supervised learning directly from real-world noisy data.
Expanding the Unified Frontier
The success of joint deblurring and interpolation has encouraged researchers to bring even more tasks into the fold:
- Super-Resolution: Jiezhang Cao and Kai Zhang proposed DAVSR (Deep Alternative Video Super-Resolution), which treats deblurring, interpolation, and super-resolution as a single optimization problem to exploit correlations between spatial and temporal degradation.
- Unknown Exposure: Real-world cameras often change exposure times dynamically. Wei Shang and Dongwei Ren developed methods to jointly interpolate and deblur even when the exact exposure settings are unknown, ensuring temporal coherence by simultaneously refining motion and exposure features.
- Domain-Specific Constraints: For specialized tasks like face video, Anh Tuan Tran and Minh Hoai introduced FineNet (Frame Interpolation and Enhancement for Face Video Deblurring), which uses facial landmarks to guide the unified reconstruction, ensuring that even under severe blur, the structural integrity of the face is preserved.
By moving away from isolated "filters" and toward unified models of how light and motion interact over time, these researchers have enabled the extraction of high-quality, high-speed video from hardware that was previously limited by the laws of physics.