Deblurring Dynamic Scenes via Spatially Varying Recurrent Neural Networks

Wenqi Ren, Jiawei Zhang, Jinshan Pan, Sifei Liu, Jimmy S. Ren, Junping Du, Xiaochun Cao, Ming-Hsuan Yang

Abstract

Deblurring images captured in dynamic scenes is challenging as the motion blurs are spatially varying caused by camera shakes and object movements. In this paper, we propose a spatially varying neural network to deblur dynamic scenes. The proposed model is composed of three deep convolutional neural networks (CNNs) and a recurrent neural network (RNN). The RNN is used as a deconvolution operator on feature maps extracted from the input image by one of the CNNs. Another CNN is used to learn the spatially varying weights for the RNN. As a result, the RNN is spatial-aware and can implicitly model the deblurring process with spatially varying kernels. To better exploit properties of the spatially varying RNN, we develop both one-dimensional and two-dimensional RNNs for deblurring. The third component, based on a CNN, reconstructs the final deblurred feature maps into a restored image. In addition, the whole network is end-to-end trainable. Quantitative and qualitative evaluations on benchmark datasets demonstrate that the proposed method performs favorably against the state-of-the-art deblurring algorithms.
BibTeX
@article{ren2022deblurring,
  author = {Wenqi Ren and Jiawei Zhang and Jinshan Pan and Sifei Liu and Jimmy S. Ren and Junping Du and Xiaochun Cao and Ming-Hsuan Yang},
  title = {{Deblurring Dynamic Scenes via Spatially Varying Recurrent Neural Networks}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2022},
  doi = {10.1109/tpami.2021.3061604},
}

Loading related content...

Figure