Non-uniform Deblurring for Shaken Images

Oliver Whyte, Josef Šivic, Andrew Zisserman, Jean Ponce

Abstract

Blur from camera shake is mostly due to the 3D rotation of the camera, resulting in a blur kernel that can be significantly non-uniform across the image. However, most current deblurring methods model the observed image as a convolution of a sharp image with a uniform blur kernel. We propose a new parametrized geometric model of the blurring process in terms of the rotational velocity of the camera during exposure. We apply this model to two different algorithms for camera shake removal: the first one uses a single blurry image (blind deblurring), while the second one uses both a blurry image and a sharp but noisy image of the same scene. We show that our approach makes it possible to model and remove a wider class of blurs than previous approaches, including uniform blur as a special case, and demonstrate its effectiveness with experiments on real images.

Introduction: Beyond Uniform Convolution

In low-light photography, camera shake is a pervasive issue. For decades, the standard computational approach to deblurring has relied on the assumption of uniform convolution. This model assumes that every pixel in an image is smeared by the exact same 2D "blur kernel." While mathematically convenient, this assumption is fundamentally flawed for real-world camera shake.

Camera shake is primarily caused by the 3D rotation (pitch, yaw, and roll) of the camera during exposure, rather than simple 2D translation. Because of the geometry of camera lenses, a rotation affects pixels differently depending on their position on the sensor. For instance, an in-plane rotation (roll) causes minimal blur at the center of the image but significant arc-like blurring at the corners. A single uniform kernel cannot resolve such spatially varying distortions.

A Parametric Geometric Model

The core contribution of this work is a parametrized geometric model that describes the blurring process as an integration of camera poses over time. Instead of estimating a 2D kernel directly, the authors estimate a 3D descriptor representing the camera's orientation $(\theta_X, \theta_Y, \theta_Z)$ throughout the exposure.

Figure 4: The 3D pose-space model $(\theta_X, \theta_Y, \theta_Z)$ and its relationship to local image patches.

Each point in this 3D pose space corresponds to a specific projective transformation (homography) of the sharp scene. The blurry image is then modeled as a weighted sum of these transformed versions. This formulation creates a bilinear system that can be solved by alternating between estimating the sharp image and the 3D blur path. To ensure physical plausibility, the authors apply $\ell_1$ regularization to the 3D kernel, enforcing the reality that a camera follows a continuous, thin path through pose-space.

Algorithm Adaptation

A key strength of this geometric model is its versatility. Rather than creating a new pipeline from scratch, the authors demonstrate how it can be integrated into existing state-of-the-art frameworks:

  • Blind Deblurring: The model was adapted to variational Bayesian and Maximum A Posteriori (MAP) approaches to estimate blur from a single image.
  • Noisy/Blurry Pairs: The model was also applied to dual-image methods, using a high-ISO noisy image to guide the restoration of a long-exposure blurry image.

Experimental Results

The proposed method was evaluated against synthetic data, standard uniform blur benchmarks, and new real-world "shaken" images. The results show that the non-uniform model is a true generalization; it performs as well as traditional methods on uniform blur (where focal length is effectively infinite) while significantly outperforming them on wide-angle or high-rotation shots.

Figure 5: Examples of spatially varying blur kernels estimated across different regions of a single shaken image.

Quantitatively, on synthetic "roll" blur, the non-uniform MAP approach achieved RMS errors roughly half those of uniform models. Qualitatively, the method successfully recovered fine details—such as text on book covers—that remained illegible or suffered from "ringing" artifacts when processed with standard convolution-based algorithms.

Conclusion and Future Work

By modeling the geometric reality of 3D camera rotation, this research provides a more accurate framework for image restoration. While the model currently assumes a static, planar scene and carries a higher computational cost than uniform methods, it represents a significant step forward in handling real-world camera shake. Future work will focus on optimizing the pose-space marginalization to reduce processing times and extending the model to handle depth-dependent parallax in dynamic scenes.

Summary updated

Abstract

Blur from camera shake is mostly due to the 3D rotation of the camera, resulting in a blur kernel that can be significantly non-uniform across the image. However, most current deblurring methods model the observed image as a convolution of a sharp image with a uniform blur kernel. We propose a new parametrized geometric model of the blurring process in terms of the rotational velocity of the camera during exposure. We apply this model to two different algorithms for camera shake removal: the first one uses a single blurry image (blind deblurring), while the second one uses both a blurry image and a sharp but noisy image of the same scene. We show that our approach makes it possible to model and remove a wider class of blurs than previous approaches, including uniform blur as a special case, and demonstrate its effectiveness with experiments on real images. 1.
BibTeX
@inproceedings{whyte2010non,
  author = {Oliver Whyte and Josef Šivic and Andrew Zisserman and Jean Ponce},
  title = {{Non-uniform Deblurring for Shaken Images}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2010},
  doi = {10.1109/cvpr.2010.5540175},
}
Figure