Single Image Defocus Deblurring via Implicit Neural Inverse Kernels

Yuhui Quan, Xin Yao, Hui Ji

Abstract

Defocus deblurring is a challenging task due to the spatially-varying nature of Point Spread Functions (PSFs) in real-world images. We present INIKNet, a lightweight and interpretable neural network that explicitly predicts per-pixel "inverse kernels." By utilizing a multi-scale dictionary parameterized by Implicit Neural Representations (INR), our model achieves state-of-the-art performance while requiring significantly fewer parameters than massive general-purpose restoration models.

Introduction

When a camera captures a scene with a shallow depth of field, objects outside the focal plane become blurry. This defocus blur is difficult to remove because it is spatially varying; every pixel has a different PSF based on its distance from the camera. Existing solutions typically fall into two categories: traditional two-step methods that estimate a defocus map, and end-to-end deep learning models. While deep learning models are powerful, they often act as black boxes that lack structural constraints or require immense computational resources.

INIKNet bridges this gap by creating a physically grounded framework that is flexible enough for real-world images. The core innovation is the Implicit Neural Inverse Kernel (INIK) framework, which forces the network to predict the mathematical "undo" operation for the blur at each pixel.

The INIKNet Architecture

INIKNet is built around a duplex scale-recurrent framework that leverages information across multiple resolutions. Unlike standard coarse-to-fine models, INIKNet passes information bi-directionally using bi-LSTMs, allowing sharp details from less-blurry regions to assist in resolving heavily blurred areas.

The INIKNet architecture features a duplex scale-recurrent framework with U-Net, Bi-LSTM, and an Inverse Filtering Module (IFM).

Key Components:

  • INR-Based Multi-Scale Dictionary: Inverse kernels are modeled as a linear combination of "atoms" from a dictionary. These atoms are parameterized by Implicit Neural Representations (MLPs), allowing the model to generate kernel atoms at any arbitrary scale with high-frequency detail. This acts as a form of structural regularization, preferring smooth, structured shapes over random noise.
  • Inverse Filtering Module (IFM): This dedicated module takes the predicted per-pixel weights and the INR-generated dictionary to perform the actual deconvolution in a single, efficient step.

Experimental Evaluation

The model was trained and evaluated on standard benchmarks including the Dual-Pixel Defocus Deblurring (DPDD) and LFDOF datasets. To test robustness, the authors performed generalization tests on the RealDOF and RTF datasets without retraining.

Quantitative comparison of INIKNet against state-of-the-art models on RealDOF and RTF datasets.

Key Results:

  • Superior Performance: On the DPDD dataset, INIKNet achieved a PSNR of 26.055 dB, outperforming previous state-of-the-art models like Restormer (25.980 dB).
  • Extreme Efficiency: INIKNet utilizes only 1.98 million parameters. In comparison, Restormer uses 26.1 million and MPRNet uses 20.1 million. This makes INIKNet approximately 13x smaller than its primary competitors while delivering superior results.
  • Robust Generalization: As shown in the quantitative results, INIKNet maintains a performance lead on unseen data (RealDOF), proving that its physics-informed inverse kernel approach is more robust than simple black-box pixel mapping.

Conclusion

INIKNet demonstrates that "physics-informed" deep learning can significantly improve image restoration tasks. By respecting the underlying optics of defocus blur and using efficient representations like INR, the model achieves state-of-the-art results with a fraction of the computational footprint of larger models. Future work may involve adapting this framework to handle directional motion blur and optimizing the computational overhead of the per-pixel filtering process.

Summary updated

Abstract

Single image defocus deblurring (SIDD) is a challenging task due to the spatially-varying nature of defocus blur, characterized by per-pixel point spread functions (PSFs). Existing deep-learning-based methods for SIDD are limited by either over-fitting due to the lack of model constraints or under-parametrization that restricts their applicability to real-world images. To address the limitations, this paper proposes an interpretable approach that explicitly predicts inverse kernels with structural regularization. Motivated by the observation that defocus PSFs within an image often have similar shapes but different sizes, we represent the inverse kernels linearly over a multi-scale dictionary parameterized by implicit neural representations. We predict the corresponding representation coefficients via a duplex scale-recurrent neural network that jointly performs fine-to-coarse and coarse-to-fine estimations. Extensive experiments demonstrate that our approach achieves excellent performance using a lightweight model.
BibTeX
@inproceedings{quan2023single,
  author = {Yuhui Quan and Xin Yao and Hui Ji},
  title = {{Single Image Defocus Deblurring via Implicit Neural Inverse Kernels}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.01158},
}
Figure