Abstract
Modern smartphone photography increasingly relies on Quad Bayer sensors to balance high resolution with low-light performance. However, these sensors introduce significant challenges for Joint Demosaicing and Denoising (JDD) due to their unique color filter array (CFA) structure. This paper introduces DRNet, a dual-encoder neural network designed to decouple noise removal from color reconstruction. By utilizing specialized Pixel Modulation and Joint Residual Blocks, DRNet achieves state-of-the-art performance, offering a 0.38dB PSNR improvement over previous methods while maintaining the efficiency required for mobile hardware.
The Quad Bayer Challenge
Most digital cameras utilize a standard Bayer Color Filter Array (CFA), where red, green, and blue pixels are arranged in a 2x2 checkerboard pattern. In contrast, Quad Bayer sensors group four pixels of the same color together in 2x2 blocks. While this design facilitates "pixel binning" for improved low-light sensitivity, it complicates the Image Signal Processor (ISP) pipeline.
The primary difficulties are twofold:
- Demosaicing: Reconstructing a full-color RGB image from sparse samples. The larger "gaps" between different colors in Quad Bayer arrays often lead to zipper artifacts and color bleeding.
- Denoising: Removing sensor noise without sacrificing fine spatial textures.
Traditional sequential processing often amplifies errors between these steps. DRNet addresses these challenges simultaneously through a unified JDD framework.
Proposed Method: DRNet Architecture
DRNet employs a Dual Encoder Network to process RAW data through two distinct paths, allowing the model to specialize in different aspects of image restoration.
Quad Bayer Encoder (QBE)
The QBE path utilizes a Pixel Modulation Block (PMB). Instead of treating the RAW image as a standard grid, the PMB regroups the 2x2 same-color clusters into four sub-maps. This preserves the physical spatial relationship of the colors before they are processed by Residual-in-Residual Dense Blocks (RRDBs) to extract high-level features.
Self-Adaptive Encoder (SAE)
The SAE path focuses on the restoration of degraded features. Its core component is the Joint Residual Block (JRB), which features two parallel branches: one for mosaic removal (color reconstruction) and one for noise removal. By decoupling these degradations, the network learns to handle them more effectively than a single-path architecture.
Self-Adaption Block (SAB)
To bridge the gap between the RAW sensor domain and the final sRGB color space, the Self-Adaption Block (SAB) uses global pooling to help the network stabilize during training and adapt features to the target domain.
Experimental Results
The researchers evaluated DRNet using both synthetic datasets (DIV2K, Flickr2K) and real-world data from the Smartphone Image Denoising Dataset (SIDD).
- Quantitative Performance: DRNet outperformed the previous state-of-the-art (SOTA) by approximately 0.38dB in PSNR across public benchmarks. On the SIDD dataset, it achieved a PSNR of 35.72, significantly higher than competitors like SAGAN (34.31).
- Visual Quality: Qualitative analysis demonstrates that DRNet preserves sharp edges and fine textures—such as text on labels or architectural grids—where other methods produce "zipper" artifacts or color moiré.
- Mobile Efficiency: A "mini" version, DRNet-M, was optimized for mobile processors like the Snapdragon 8 Gen 2. It achieved a processing time of just 49ms for a 192x192 patch, making it a viable candidate for real-world smartphone ISPs where Transformer-based models often face memory constraints.
Conclusion
DRNet demonstrates that a "divide and conquer" approach to joint demosaicing and denoising is highly effective for Quad Bayer sensors. By separating the estimation of noise and mosaic residuals within a dual-encoder framework, the model achieves superior image quality and computational efficiency, paving the way for more robust mobile imaging pipelines.
Summary updated