Abstract
Deep learning based image denoising methods have been extensively investigated. In this paper, attention mechanism enhanced kernel prediction networks (AME-KPNs) are proposed for burst image denoising, in which, nearly cost-free attention modules are adopted to first refine the feature maps and to further make a full use of the inter-frame and intra-frame redundancies within the whole image burst. The proposed AME-KPNs output per-pixel spatially-adaptive kernels, residual maps and corresponding weight maps, in which, the predicted kernels roughly restore clean pixels at their corresponding locations via an adaptive convolution operation, and subsequently, residuals are weighted and summed to compensate the limited receptive field of predicted kernels. Simulations and real-world experiments are conducted to illustrate the robustness of the proposed AME-KPNs in burst image denoising.
Introduction
Capturing high-quality images in low-light environments remains a significant challenge due to the prevalence of shot and read noise. Modern mobile photography addresses this through burst imaging—capturing a rapid sequence of frames and merging them to improve the signal-to-noise ratio. However, effectively denoising these bursts requires balancing noise suppression with detail preservation.
Traditional Kernel Prediction Networks (KPNs) attempt to solve this by predicting a unique filter (kernel) for every pixel. However, they are often limited by a fundamental trade-off: small kernels preserve sharp details but struggle with heavy noise or frame misalignment, while larger kernels effectively denoise but often introduce unwanted blur. The proposed Attention Mechanism Enhanced Kernel Prediction Network (AME-KPN) breaks this trade-off by integrating attention mechanisms and a novel "kernel plus residual" prediction strategy.
Methodology
The AME-KPN architecture is built upon a modified U-Net structure, enhanced with two primary innovations: Dual Attention Modules and a hybrid prediction output.
Dual Attention Modules
At each level of the decoder, the network incorporates both Channel and Spatial Attention. Channel Attention prioritizes the most informative feature maps, while Spatial Attention identifies critical regions within the image. This dual approach allows the model to better exploit inter-frame redundancies (similarities across the burst) and intra-frame redundancies (patterns within a single frame) without significant computational overhead.
Kernel and Residual Prediction
Unlike standard KPNs that only output kernels, AME-KPN predicts three components for every pixel:
- Spatially-adaptive kernels: For local denoising via adaptive convolution.
- Residual maps: To capture high-frequency details and compensate for the limited receptive field of the kernels.
- Weight maps: To intelligently blend the kernel-denoised output with the residual map.
Training and Datasets
The model is trained using an annealed loss function that transitions from per-frame denoising to full-burst aggregation. Evaluation was conducted on a synthetic dataset derived from Adobe-5K (simulating realistic camera noise and motion) and the PolyU real-world dataset to ensure generalization.
Results and Discussion
AME-KPN consistently outperforms standard KPN and Multi-Kernel Prediction Network (MKPN) baselines. Quantitatively, at low noise levels (Gain 1), AME-KPN achieved a PSNR of 39.92 dB, representing a ~2 dB improvement over standard 5x5 KPNs.
Qualitative results highlight the model's ability to preserve fine textures and sharp edges. For instance, in real-world lens cap tests, AME-KPN successfully restored legible text where other methods produced significant blurring or artifacts.
Ablation studies confirmed that both the attention modules and the residual map are critical for performance. Removing either component led to a measurable drop in PSNR, with the full model providing nearly a 1 dB gain over the baseline.
Conclusion
The AME-KPN framework demonstrates a robust approach to burst image denoising by leveraging attention mechanisms to implicitly align frames and aggregate information. By combining local kernels with global residual compensation, the model effectively overcomes the receptive field limitations of traditional KPNs. Future work may involve extending this "kernel + residual" framework to other restoration tasks such as super-resolution and video deblurring.
Summary updated