Abstract
Denoising is a fundamental imaging problem. Versatile but fast filtering has been demanded for mobile camera systems. We present an approach to multiscale filtering which allows real-time applications on low-powered devices. The key idea is to learn a set of kernels that upscales, filters, and blends patches of different scales guided by local structure analysis. This approach is trainable so that learned filters are capable of treating diverse noise patterns and artifacts. Experimental results show that the presented approach produces comparable results to state-of-the-art algorithms while processing time is orders of magnitude faster.
Introduction
Image denoising is a fundamental challenge in computer vision. While modern methods like BM3D or deep Convolutional Neural Networks (CNNs) produce high-quality results, their computational weight often makes them unsuitable for real-time applications on mobile cameras and low-power devices. Removing heavy noise typically requires large receptive fields, which traditionally leads to a quadratic increase in processing time. This paper addresses this gap by introducing a multiscale denoising algorithm that achieves a broad receptive field using small, efficient filters, enabling high-performance denoising at speeds orders of magnitude faster than current state-of-the-art techniques.
Methodology
The proposed method extends the "Best Linear Adaptive Enhancement" (BLADE) framework into a multiscale pipeline. The architecture relies on spatially-adaptive filtering, which is highly parallelizable and hardware-friendly.
Spatially-Adaptive Filtering
The algorithm analyzes the local neighborhood of every pixel to compute a structure tensor, determining local gradient orientation, strength, and coherence. Notably, this analysis is performed jointly across all RGB channels to preserve color-specific edges that might be missed in luminance-only processing. Based on these structural features, a small linear filter (e.g., 5x5) is selected from a pre-trained filterbank and applied to the patch.
Multiscale Cascade
To handle large-scale noise without the overhead of large filters, the algorithm creates an image pyramid. Starting from the coarsest resolution, learned filters are applied that simultaneously upscale the coarser image, denoise the current level, and blend them together. This cascading approach allows a small 5x5 filter to effectively cover a much larger area of the original image (e.g., 20x20 pixels after two downsampling steps), providing the necessary context for heavy noise removal at a fraction of the cost.
Experimental Results
The filterbanks were trained on a dataset of 2.23 billion pixels from high-quality internet images, augmented with synthetic noise and geometric transformations. Evaluation was performed using the Berkeley dataset.
Speed and Efficiency
The standout feature of this approach is its processing speed. On a 1-megapixel image, the method takes just 0.038 seconds, representing a 400x to 500x speedup over methods like C-BM3D (21.9s) and TNRD (16.5s). On a 2017 Google Pixel phone, the GPU implementation achieves a throughput of 188 Megapixels per second (MP/s), making it highly viable for real-time mobile photography.
Quality Trade-offs
The algorithm achieves competitive image quality. At a noise standard deviation of $\sigma=25$, the method reaches a PSNR of 29.58 dB, which is close to the 30.18 dB achieved by the much slower C-BM3D. This small quality difference is a favorable trade-off for the massive gains in efficiency.
Key Insights and Conclusion
The efficiency of this method stems from its "shallow" architecture, which utilizes simple additions and multiplications with few sequential dependencies. This makes it ideal for hardware vectorization and parallelization compared to deep CNNs. Furthermore, joint RGB analysis ensures that structural details are preserved across all color channels.
Future work will focus on integrating automatic noise estimation to handle non-uniform noise levels and developing sensor-specific models to further optimize performance for individual camera hardwares.
Summary updated