Attention Mechanism Enhanced Kernel Prediction Networks for Denoising of Burst Images

Bin Zhang, Shenyao Jin, Yili Xia, Yongming Huang, Zixiang Xiong

Abstract

Deep learning based image denoising methods have been extensively investigated. In this paper, attention mechanism enhanced kernel prediction networks (AME-KPNs) are proposed for burst image denoising, in which, nearly cost-free attention modules are adopted to first refine the feature maps and to further make a full use of the inter-frame and intra-frame redundancies within the whole image burst. The proposed AME-KPNs output per-pixel spatially-adaptive kernels, residual maps and corresponding weight maps, in which, the predicted kernels roughly restore clean pixels at their corresponding locations via an adaptive convolution operation, and subsequently, residuals are weighted and summed to compensate the limited receptive field of predicted kernels. Simulations and real-world experiments are conducted to illustrate the robustness of the proposed AME-KPNs in burst image denoising.

Introduction

Capturing high-quality images in low-light environments remains a significant challenge due to the prevalence of shot and read noise. Modern mobile photography addresses this through burst imaging—capturing a rapid sequence of frames and merging them to improve the signal-to-noise ratio. However, effectively denoising these bursts requires balancing noise suppression with detail preservation.

Traditional Kernel Prediction Networks (KPNs) attempt to solve this by predicting a unique filter (kernel) for every pixel. However, they are often limited by a fundamental trade-off: small kernels preserve sharp details but struggle with heavy noise or frame misalignment, while larger kernels effectively denoise but often introduce unwanted blur. The proposed Attention Mechanism Enhanced Kernel Prediction Network (AME-KPN) breaks this trade-off by integrating attention mechanisms and a novel "kernel plus residual" prediction strategy.

Methodology

The AME-KPN architecture is built upon a modified U-Net structure, enhanced with two primary innovations: Dual Attention Modules and a hybrid prediction output.

Dual Attention Modules

At each level of the decoder, the network incorporates both Channel and Spatial Attention. Channel Attention prioritizes the most informative feature maps, while Spatial Attention identifies critical regions within the image. This dual approach allows the model to better exploit inter-frame redundancies (similarities across the burst) and intra-frame redundancies (patterns within a single frame) without significant computational overhead.

Kernel and Residual Prediction

Unlike standard KPNs that only output kernels, AME-KPN predicts three components for every pixel:

  1. Spatially-adaptive kernels: For local denoising via adaptive convolution.
  2. Residual maps: To capture high-frequency details and compensate for the limited receptive field of the kernels.
  3. Weight maps: To intelligently blend the kernel-denoised output with the residual map.
The architecture of the proposed AME-KPN, featuring dual attention modules and a kernel-plus-residual prediction strategy.

Training and Datasets

The model is trained using an annealed loss function that transitions from per-frame denoising to full-burst aggregation. Evaluation was conducted on a synthetic dataset derived from Adobe-5K (simulating realistic camera noise and motion) and the PolyU real-world dataset to ensure generalization.

Results and Discussion

AME-KPN consistently outperforms standard KPN and Multi-Kernel Prediction Network (MKPN) baselines. Quantitatively, at low noise levels (Gain 1), AME-KPN achieved a PSNR of 39.92 dB, representing a ~2 dB improvement over standard 5x5 KPNs.

Qualitative results highlight the model's ability to preserve fine textures and sharp edges. For instance, in real-world lens cap tests, AME-KPN successfully restored legible text where other methods produced significant blurring or artifacts.

Qualitative comparison of denoising results on a lens cap image. AME-KPN (f) preserves sharp text and fine details better than competing methods.

Ablation studies confirmed that both the attention modules and the residual map are critical for performance. Removing either component led to a measurable drop in PSNR, with the full model providing nearly a 1 dB gain over the baseline.

Conclusion

The AME-KPN framework demonstrates a robust approach to burst image denoising by leveraging attention mechanisms to implicitly align frames and aggregate information. By combining local kernels with global residual compensation, the model effectively overcomes the receptive field limitations of traditional KPNs. Future work may involve extending this "kernel + residual" framework to other restoration tasks such as super-resolution and video deblurring.

Summary updated

Abstract

Deep learning based image denoising methods have been extensively investigated. In this paper, attention mechanism enhanced kernel prediction networks (AME-KPNs) are proposed for burst image denoising, in which, nearly cost-free attention modules are adopted to first refine the feature maps and to further make a full use of the inter-frame and intra-frame redundancies within the whole image burst. The proposed AME-KPNs output per-pixel spatially-adaptive kernels, residual maps and corresponding weight maps, in which, the predicted kernels roughly restore clean pixels at their corresponding locations via an adaptive convolution operation, and subsequently, residuals are weighted and summed to compensate the limited receptive field of predicted kernels. Simulations and real-world experiments are conducted to illustrate the robustness of the proposed AME-KPNs in burst image denoising.
BibTeX
@inproceedings{zhang2020attention,
  author = {Bin Zhang and Shenyao Jin and Yili Xia and Yongming Huang and Zixiang Xiong},
  title = {{Attention Mechanism Enhanced Kernel Prediction Networks for Denoising of Burst Images}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=HWl4xg-doZt},
}
Figure