Abstract
Leading denoising methods such as 3D block matching (BM3D) are patch-based. However, they can suffer from frequency domain artefacts and require to specify explicit noise models. We present a patch-based method that avoids these drawbacks. It combines a simple and fast patch reordering with a non-linear smoothing. The smoothing rewards both patch and pixel similarities in a multiplicative way. We perform experiments on real world images with additive white Gaussian noise (AWGN), and on electron microscopy data with a more general additive noise model. Our filter outperforms BM3D in 77% of the experiments, with improvements of up to 29% with respect to the mean squared error.
Introduction
Image denoising aims to recover a clean image from a noisy observation while preserving critical structures like edges and textures. While patch-based methods such as 3D Block Matching (BM3D) and Non-local Bayes (NLB) represent the current state-of-the-art, they possess significant limitations. Specifically, they often introduce visual artifacts—hallucinated patterns or blurring—due to the accidental mixing of dissimilar image regions. Furthermore, these methods typically rely on rigid Additive White Gaussian Noise (AWGN) models, which limits their effectiveness on real-world scientific data, such as electron microscopy, where noise characteristics are more complex.
The NFPR Framework
The proposed method, Non-linear Filtering on Fast Patch Reorderings (NFPR), addresses these issues through a two-step process:
- Fast Patch Reordering: Previous reordering methods attempted to solve the computationally expensive Traveling Salesman Problem to find a global ordering of patches. NFPR simplifies this by performing a fast local search. For each pixel, similarity is calculated within a disc-shaped neighborhood, and pixels are sorted based on this similarity. This approach provides sufficient structure for filtering while remaining computationally efficient.
- Multiplicative Non-linear Smoothing: The core innovation is a filter that employs multiplicative weights. The weight for a pixel pair is the product of their patch similarity and their tonal (pixel value) similarity. This multiplicative approach acts as a strict gatekeeper: if either the local neighborhoods or the individual pixel values are dissimilar, the weight drops to near zero. This prevents the "diffusion" of incorrect information that typically generates artifacts in other methods.
Experimental Evaluation
The researchers evaluated NFPR against standard benchmark images (e.g., Lena, Bridge, House, Peppers) and challenging electron microscopy data of a yeast cell ribosome.
In standard benchmarks, NFPR outperformed BM3D in 77% of experiments, achieving up to a 29% reduction in Mean Squared Error (MSE). Visually, NFPR results are characterized by cleaner edges and a lack of the "smudging" or "rippling" artifacts common in BM3D and NLB.
On electron microscopy data, NFPR's model-agnostic nature allowed it to handle complex noise more effectively. The Fourier Ring Correlation (FRC) plot confirms that NFPR preserves information across higher frequencies better than BM3D or NLB, which struggle with the non-standard noise distribution.
Computational Efficiency
NFPR is highly efficient. On a GPU, it can denoise a 512x512 image in approximately 6.5 seconds. This is a massive improvement over previous patch-reordering techniques, which could take nearly two hours for the same task on a CPU. This speed makes the method practical for high-throughput scientific imaging.
Conclusion
NFPR provides a robust, efficient, and artifact-resistant alternative to traditional patch-based denoising. By combining fast local reordering with multiplicative non-linear filtering, it achieves superior results on both synthetic and real-world noise models. Future work will explore applying these principles to other inverse problems like deblurring and super-resolution.
Summary updated