Temporal Unfolding from Motion Blurred Images

67 concepts47 papers2 proposals

Traditional computer vision pipelines often treat motion blur as a destructive artifact—a chaotic loss of high-frequency data caused by a camera shutter staying open too long while a scene changes. Under this view, deblurring is an ill-posed restoration problem where lost details must be guessed or hallucinated.

However, a physical analysis of image formation reveals a different perspective: motion blur is not corrupted data, but integrated data. It acts as a continuous temporal sensor that compresses a high-speed event into a single spatial frame. By understanding the forward physics of this integration, we can invert the process. This allows us to perform temporal unfolding—extracting high-speed video sequences, dense optical flow, and complex 3D trajectories from a single motion-blurred exposure.


Blur as a Temporal Integrator

To understand why temporal unfolding is possible, we must look at how a camera sensor accumulates light. During an exposure window $[0, \tau]$, the sensor acts as a temporal integrator. The resulting blurred pixel value $B$ at a spatial coordinate is the average of the latent sharp scene states $I(t)$ over time:

$$B = \frac{1}{\tau} \int_0^{\tau} I(t) \, dt$$

Because the camera continuously records the scene throughout the exposure, the temporal history is preserved, mapped directly onto the spatial dimensions of the sensor. As shown by Benedicte Bascle et al., treating motion blur as a linear transformation across time allows us to leverage this temporal redundancy, turning a smear into a set of geometric constraints that facilitate the recovery of high-frequency details.


The Spatial-Temporal Mapping

How exactly does time map to space within a single frame? When an object moves, its trajectory leaves a distinct signature or "streak" on the sensor. The boundaries, transparency, and shape of this streak act as a physical roadmap of the motion.

Boundary States

The sharp edges at the very ends of a blur streak represent the exact states of the object at the moments the shutter opened and closed. Chaoyue Wang et al. leverage these sharp boundary states as physical constraints to define the start and end of an exposure period, allowing intermediate motion to be interpolated without predefined temporal priors. Similarly, Dawit Mureja Argaw et al. train neural networks to decode these spatial smears into distinct feature sets representing the start and end of the motion period, enabling dense optical flow estimation from a single image.

  Exposure Starts (t=0)                      Exposure Ends (t=τ)
         |                                          |
         ▼                                          ▼
  [ Sharp Boundary ] ======= Spatial Smear ======= [ Sharp Boundary ]
                            ▲             ▲
                            |             |
                 Trajectory Curvature    Dwell Time (Density)

Trajectory and Intrinsic Dynamics

The internal structure of the blur encodes complex, non-uniform dynamics. If an object accelerates, rotates, or changes shape, the spatial density and transparency of the blur streak alter accordingly. Jochen Gast et al. demonstrate that non-uniform 2D spatial smears contain sufficient information to estimate complex parametric motion like rotation and shear from a single image.

For Fast Moving Objects (FMOs)—objects that travel a distance greater than their own size within a single exposure—the streak becomes semi-transparent. Denys Rozumnyi et al. show that the shape and transparency of these streaks provide a rich signal to solve space-variant deconvolution problems, recovering both the object's sharp appearance and its 3D rotation axis.


Unrolling the Temporal Container

To turn these spatial cues back into a video, the integrated path within a single exposure must be partitioned into discrete temporal segments. This process is known as temporal unfolding.

Jan Kotera demonstrates that by slicing a motion-blurred path into sequential segments, a system can reconstruct high-resolution snapshots at time intervals significantly shorter than the camera's exposure time. This effectively turns standard video sensors into sub-frame temporal integrators.

With the advent of deep learning, networks can learn to separate scene content from motion dynamics to automate this unrolling. Frameworks proposed by Kuldeep Purohit et al. and Denys Rozumnyi et al. treat the static blurred image as a temporal summary, using recurrent or generative architectures to unpack the continuous exposure into a sequential video.

This inversion is not restricted to 2D pixel space; it can be extended to recover 3D trajectories and articulated poses:

  • Articulated Tracking: JoonKyu Park et al. extract sequential 3D hand meshes from a single blurry frame by treating the blur as a temporal container of motion rather than noise. Yiming Zhao et al. apply a similar forward model to perform 3D human pose tracking from single exposures.
  • 3D Trajectories: By modeling the physics of motion blur in the 3D domain, Denis Rozumny et al. reconstruct the 3D shape, texture, and 6-DoF trajectory of fast-moving objects, capturing how different facets of a rotating object contribute to the final blurred streak.

Leveraging Shutter Mechanics and Active Illumination

The precision of temporal unfolding can be enhanced by exploiting the physical mechanics of the camera shutter or by actively modifying the illumination during the exposure.

Shutter-Induced Gradients

Instead of exposing the entire sensor simultaneously, rolling shutters capture rows sequentially. Vijay Rengarajan demonstrates that the resulting spatial distortions (such as skew and curvature) function as informative cues that map row index to absolute time, allowing the recovery of continuous camera movement from a single frame.

In hybrid exposure modes like Global Reset Release (GRR), the exposure time varies across rows, creating row-dependent blur magnitudes. Xiang Ji shows that this spatial distribution of varying blur levels provides enough structural information to match the performance of multi-frame video deblurring techniques using only a single capture.

Active Temporal Encoding

Rather than relying on passive light accumulation, systems can actively encode temporal information into the spatial blur. Yuki Shiba introduces a method that projects a sequence of distinct structured light patterns at a high frequency within a single camera exposure.

Time (t):          |-- t1 --|-- t2 --|-- t3 --|
Projected Pattern: | Pattern A | Pattern B | Pattern C |
                   \____________ ______________/
                                V
Capture:             [ Single Spatial Blur Streak ]
                     (Decodable via Pattern Overlays)

Because each temporal segment of the resulting spatial blur corresponds to a specific projected pattern, the blur becomes a highly structured, decodable roadmap. This technique bypasses the frame-rate limitations of standard hardware, recovering high-speed 3D shape sequences at up to 1,800 fps from a single blurred frame.


Differentiable Re-blurring in 3D Scene Representations

The principle of the temporal integral has also transformed how 3D scene representations—such as Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting—are trained on real-world, imperfect data.

Instead of attempting to deblur images before optimization, modern frameworks incorporate the physical blur formation process directly into the training loop via differentiable re-blurring. To optimize a scene from blurry inputs, Byeonghyeon Lee et al. and Lingzhe Zhao et al. model a blurred image as the temporal average of multiple sharp virtual views rendered along a continuous camera trajectory during the exposure window.

$$\text{Loss} = \left\| B_{\text{observed}} - \frac{1}{N}\sum_{i=1}^N \mathcal{R}(\mathbf{x}, t_i) \right\|_2$$

Where $\mathcal{R}(\mathbf{x}, t_i)$ represents the rendered virtual view at sub-frame time step $t_i$. By supervising the model based on the integrated output of these virtual views, the underlying optimization naturally converges on a perfectly sharp, high-fidelity 3D representation. This demonstrates the core utility of the paradigm: by treating blur as a physical integral rather than data loss, a common image artifact becomes a powerful source of spatial and temporal information.

Key Foundational Work

Motion Deblurring and Super-resolution from an Image Sequence (1996) — contributed 2 concept elements to this direction.
Detecting Salient Motion by Accumulating Directionally-Consistent Flow (1999) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 48 papers
  2. Version 1 · 2026-10-04 · 69 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_temporal_super_resolution_from_motion_bl_a2fd21e5,
  title = {temporal super-resolution from motion blur},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-super-resolution-from-motion-blur/rdr_a2fd21e5e5b2434db9121d0d9639ee9b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_temporal_super_resolution_from_motion_bl_a2fd21e5_v2,
  title = {temporal super-resolution from motion blur},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-super-resolution-from-motion-blur/rdr_a2fd21e5e5b2434db9121d0d9639ee9b/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake