K3DN: Disparity-Aware Kernel Estimation for Dual-Pixel Defocus Deblurring

Yan Yang, Liyuan Pan, Liu Liu 0009, Miaomiao Liu 0001

Abstract

Defocus blur in dual-pixel (DP) images is spatially-varying and challenging to correct without precise disparity information. We present K3DN, an efficient framework that explicitly models the relationship between image disparity and blur kernels using a trainable kernel dictionary. By incorporating a Sharp Region Preservation (SRP) module and physics-based symmetry constraints, K3DN achieves state-of-the-art performance on standard DP datasets while maintaining significantly lower computational overhead compared to existing transformer-based models.

Introduction

Defocus blur occurs when scene points fall outside a camera's depth-of-field. Modern Dual-Pixel (DP) sensors, common in smartphones and DSLRs, provide a unique advantage by splitting each pixel into left and right photodiodes. The resulting disparity between these views is directly proportional to the amount of blur. However, existing deblurring methods often fail to fully exploit this physical relationship, either treating the problem as a black-box end-to-end task or requiring hard-to-obtain ground-truth disparity maps. K3DN addresses these issues by learning physics-based kernels directly from DP data without needing extra supervision.

Methodology

The K3DN framework consists of three primary components designed to handle spatially-varying blur efficiently:

  1. Disparity-aware Deblur Module: Instead of direct image restoration, the model estimates a disparity feature map that queries a trainable kernel set. This dictionary of potential blur patterns allows the model to construct a spatially-varying blur estimate. Kernels are constrained to be horizontally symmetric, reflecting the physical properties of DP sensors.
  2. Sharp Region Preservation (SRP) Module: To prevent artifacts in already-sharp regions, the SRP module identifies in-focus areas using an attention mechanism. It "mines" original sharp features to ensure they are preserved during the deblurring process, preventing the introduction of artificial noise or ringing.
  3. Reblurring Regularization: An unsupervised training loop is employed where the model re-blurs its estimated sharp image to match the original input, helping it learn accurate kernels without explicit disparity supervision.
The K3DN architecture featuring the PSF block for kernel estimation and the SRP block for preserving in-focus regions.

Datasets

Evaluation was conducted on four standard DP datasets:

  • Synthetic: DDD-syn (5,000 pairs based on NYU RGBD) and RDPD (10,115 pairs synthesized using a Butterworth filter).
  • Real-world: DPD-blur (350 training/76 testing pairs from a Canon EOS 5D Mark IV) and DPD-disp (100 images used for qualitative evaluation).

Results and Discussion

K3DN demonstrates a superior balance between restoration quality and computational efficiency. On the real-world DPD-blur dataset, the "Large" version of K3DN achieved a PSNR of 27.06 dB, outperforming the previous state-of-the-art model, Restormer (26.66 dB).

Notably, the lightweight version of K3DN matches state-of-the-art performance with only 5.00M parameters and 1033G FLOPs, a fraction of the resources required by Restormer (26.13M parameters and 4458G FLOPs). Qualitative results confirm that K3DN restores sharper text and finer details compared to competing methods, which often leave "ghosting" or blurry edges.

Comparison of PSNR vs. Parameters (left) and PSNR vs. FLOPs (right) showing K3DN's efficiency relative to other models.

Conclusion

K3DN provides a robust solution for DP defocus deblurring by integrating physical priors into a deep learning framework. By focusing on both kernel estimation and sharp region preservation, it avoids common pitfalls like ringing artifacts and excessive computational cost. Future work will explore handling complex optical aberrations in low-cost lenses and extending the framework to video sequences where temporal consistency is key.

Summary updated

Abstract

The dual-pixel (DP) sensor captures a two-view image pair in a single snapshot by splitting each pixel in half. The disparity occurs in defocus blurred regions between the two views of the DP pair, while the in-focus sharp regions have zero disparity. This motivates us to propose a K3DN framework for DP pair deblurring, and it has three modules: i) a disparity-aware deblur module. It estimates a disparity feature map, which is used to query a trainable kernel set to estimate a blur kernel that best describes the spatially-varying blur. The kernel is constrained to be symmetrical per the DP formulation. A simple Fourier transform is performed for deblurring that follows the blur model; ii) a reblurring regularization module. It reuses the blur kernel, performs a simple convolution for reblurring, and regularizes the estimated kernel and disparity feature unsupervisedly, in the training stage; iii) a sharp region preservation module. It identifies in-focus regions that correspond to areas with zero disparity between DP images, aims to avoid the introduction of noises during the deblurring process, and improves image restoration performance. Experiments on four standard DP datasets show that the proposed K3DN outperforms state-of-the-art methods, with fewer parameters and flops at the same time.
BibTeX
@inproceedings{yang2023k,
  author = {Yan Yang and Liyuan Pan and Liu Liu and Miaomiao Liu},
  title = {{K3DN: Disparity-Aware Kernel Estimation for Dual-Pixel Defocus Deblurring}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.01274},
}
Figure