Introduction
In modern digital photography, capturing high-quality images in low-light conditions remains a significant challenge. Most consumer cameras utilize a Color Filter Array (CFA), typically the Bayer pattern, which records only one color (Red, Green, or Blue) per pixel. To produce a full-color image, the camera must perform two critical tasks: denoising to remove sensor noise and demosaicking to interpolate the missing color values.
Traditionally, these steps are performed sequentially. However, this approach is prone to error; demosaicking can amplify existing noise, while denoising can destroy the fine structural details needed for accurate interpolation. While Joint Denoising and Demosaicking (JDD) methods have emerged to solve this, they often struggle with the complex, signal-dependent noise of real-world sensors. This paper introduces GCP-Net, a framework designed for JDD of burst images (JDD-B), which leverages a sequence of short-exposure frames to reconstruct a single high-fidelity output.
The Green Channel Advantage
The core innovation of GCP-Net is the Green Channel Prior (GCP). In a standard Bayer filter, green pixels are sampled at twice the density of red or blue pixels (50% vs. 25%). This higher sampling rate, combined with the spectral sensitivity of most sensors, results in the green channel having a significantly higher signal-to-noise ratio (SNR) and better structural preservation than its counterparts.
As shown in the SNR analysis above, the green channel consistently outperforms the red and blue channels across various real-world image samples. GCP-Net exploits this "gold standard" channel to guide the restoration of the noisier red and blue data.
GCP-Net Architecture
GCP-Net is composed of three specialized modules that integrate the Green Channel Prior throughout the reconstruction pipeline:
- Intra-frame Module: This module extracts features from individual frames using Green Channel Attention (GCA) blocks. These blocks use the dense green channel features to provide pixel-wise scaling and bias adjustments, effectively "guiding" the feature extraction for the sparser red and blue channels.
- Inter-frame Module: To utilize the information across a burst, frames must be aligned to compensate for camera shake or subject motion. Alignment is notoriously difficult in noisy data. GCP-Net solves this by estimating motion offsets specifically from the cleaner green channel features. It then employs Deformable Convolutions (DConv) and a ConvLSTM to aggregate these features while maintaining temporal consistency.
- Merge Module: The final stage aggregates aligned features and performs GCP-guided adaptive upsampling. By using the high-resolution structure of the green channel as a template, the network ensures that the final RGB output maintains sharp edges and avoids common artifacts like "zippering" or moiré patterns.
Experimental Results
The authors evaluated GCP-Net against state-of-the-art burst restoration methods like KPN and EDVR. Quantitatively, GCP-Net achieved a significant PSNR gain of approximately 2.0dB over EDVR-based baselines on the REDS4 dataset in high-noise scenarios.
Qualitatively, the impact of the GCP is evident. As seen in the comparison above, the full model preserves fine textures and sharp edges that are lost or blurred in versions of the model that do not utilize the green channel prior. The method effectively handles large motions in night-shot bursts without the "ghosting" artifacts common in other burst-processing techniques.
Conclusion
GCP-Net demonstrates that the inherent properties of the Bayer sensor can be leveraged to significantly improve image reconstruction. By treating the green channel as a structural anchor, the model achieves superior denoising and demosaicking performance on real-world burst images. Future work may involve optimizing the architecture for mobile ISPs and extending the prior to non-Bayer CFA patterns like X-Trans.
Summary updated