When a camera shakes or an object moves across a frame, computer vision algorithms attempt to reconstruct the underlying physical reality. In blind image deblurring, this means estimating a blur kernel (the point spread function, or PSF) that represents the camera's path. In optical or scene flow estimation, it means mapping the velocity vectors of pixels across frames.
Naively, these tasks are framed as high-dimensional optimization problems where every pixel or kernel element can be solved independently. However, treating these elements as discrete, unconstrained variables ignores a fundamental truth: motion happens in a physical world. A shaking camera follows a smooth, unbroken trajectory through space; a moving object cannot instantly teleport or fracture into chaotic, independent pixel movements.
By encoding physical continuity directly into optimization objectives, researchers have transformed highly ill-posed estimation problems into robust, physically plausible systems.
The Failure of Discrete Abstractions
To understand why physical continuity priors work, we must first look at what happens without them. Many traditional image restoration techniques treat a blur kernel as a simple 2D matrix of independent values. As Wu et al. point out, this mathematical abstraction completely fails to account for the temporal continuity of camera trajectories.
If you attempt to solve for a blur kernel using standard sparsity priors (such as a Hyper-Laplacian prior), the optimization algorithm knows that most of the matrix should be zero, but it doesn't understand how the non-zero elements relate to one another. As observed by Pan et al., this mismatch frequently causes the algorithm to converge on non-physical solutions—resulting in an estimated kernel that looks like a scattered collection of disconnected, noisy dots rather than a smooth path.
Standard Sparsity Prior (Noisy/Disconnected)
. • . . • . .
. . • . . • .
• . . . . . •
Continuity-Preserving Prior (Connected Trajectory)
. ▬▬ . . . .
. . . ╲ . . .
. . . . ▬▬ .
A parallel problem occurs in motion estimation. When tracking optical flow across a video, optimizing purely for pixel matching (brightness constancy) leaves the algorithm blind in regions with low texture or uniform color. Without an explicit constraint to enforce visual coherence across the spatial domain, neighboring pixels will be assigned erratic, conflicting motion vectors, destroying the integrity of the flow field (Gori et al.).
Deblurring the Blur via Gradient Regularization
To force an optimization algorithm to draw a continuous trajectory instead of scattering isolated dots, we have to change what we penalize. The core insight, developed across several works by Pan et al., is to shift the regularization constraint from the kernel’s pixel values to the kernel’s gradients.
By applying an $L_0$-like regularization directly to the gradients of the blur kernel, the objective function explicitly counts and penalizes the number of non-zero differences between adjacent kernel elements. Isolated noise dots are heavily penalized because each dot creates a massive number of sharp gradient transitions with its empty surroundings. Conversely, a smooth, continuous path minimizes gradient changes along its trajectory. This continuity-preserving prior forces the estimated kernel to form a connected physical path, making it highly robust against noise (Pan et al.).
This trajectory-centric view allows algorithms to resolve complex blurs by treating the kernel as a thin 1D path cutting through a 2D or 3D space:
- Wu et al. decompose the blur kernel into physically meaningful components—trajectory, intensity, and point spread function—dramatically reducing the optimization search space and avoiding local minima traps.
- Shao et al. combine sparsity and smoothness constraints directly on the kernel to eliminate isolated noise points, ensuring a physically plausible PSF recovery.
- Sivic et al. extend this to 3D, applying sparsity constraints to a camera orientation kernel to enforce the physical reality that camera motion follows a continuous, thin trajectory through pose-space during an exposure.
- Zhu et al. leverage the inherent spatial sparsity of these 1D paths, using sparsity-promoting regularization to "deblur the blurs" and sharpen fuzzy initial trajectory approximations.
Spatial Coherence in Motion Fields
Physical continuity constraints are equally powerful when applied across the spatial dimensions of an image. Because real-world motion is caused by solid objects or moving cameras, the motion observed at one pixel is highly correlated with the motion of its neighbors.
In optical flow and scene flow estimation, this concept dates back to foundational frameworks like the "Skin and Bones" model introduced by Ju et al.. This approach enforces spatial coherence by applying a smoothness constraint across the parameters of neighboring local motion models, penalizing abrupt discrepancies to ensure a globally consistent motion field.
In modern deep learning, where ground-truth motion data can be incredibly difficult to obtain, this classical principle serves as a vital guiding light. Harley et al. demonstrated that by integrating spatial smoothness constraints directly into an unsupervised training objective, convolutional neural networks can successfully resolve motion ambiguities in textureless regions without requiring any human-annotated labels.
However, enforcing spatial smoothness introduces a delicate engineering trade-off. As Bendig et al. observe in self-supervised scene flow prediction, while spatial smoothness regularization stabilizes training in low-texture areas, it can inadvertently penalize the sharp, legitimate discontinuities that occur at the boundaries of moving objects, leading to blurred edges.
Smoothness Prior Over-Correction
[ Object Moving Left ] ➔ [ Blurred Boundary ] ➔ [ Background Stationary ]
Mapping Non-Uniform and Row-by-Row Motion
When camera motion varies across a single frame, physical continuity acts as the glue that holds the image reconstruction together.
In real-world handheld photography, camera shake is rarely uniform; 3D rotations cause the blur kernel to change depending on the pixel's location on the sensor. To handle this, Harmeling et al. apply a regularization term that constrains the blur kernels of adjacent image regions to be mathematically similar. This constraint directly mirrors the continuous nature of camera movement, preventing erratic kernel variations in low-texture areas. Similarly, Mai et al. use Gaussian Conditional Random Fields to model the correlations between neighboring pixels in a blur kernel field, preventing the artifacts that inevitably arise from independent, element-wise pixel estimation.
This principle is also essential for correcting rolling shutter artifacts, which occur when CMOS sensors capture an image row-by-row while the camera is moving. Zeng demonstrates that incorporating Gaussian-weighted blending between estimated image blocks ensures that the reconstructed camera motion remains physically continuous. This constraint prevents the formation of jarring visual artifacts, proving that enforcing physical motion trajectories is a prerequisite for realistic image restoration.
Key Foundational Work
Space-Variant Single-Image Blind Deconvolution for Removing Camera Shake (2010) — contributed 2 concept elements to this direction.
Skin and Bones: Multi-layer, Locally Affine, Optical Flow and Regularization with Transparency (1996) — contributed 1 concept elements to this direction.