Abstract
Median2Median (M2M) is a novel zero-shot denoising framework designed to remove structured, correlated noise—such as stripes or banding—from a single image without requiring external training data. By combining directional interpolation with robust median filtering, M2M generates pseudo-independent image pairs that enable the Noise2Noise training strategy to handle complex, real-world noise patterns for the first time. Experimental results demonstrate that M2M significantly outperforms existing zero-shot methods and classic algorithms like BM3D in structured noise scenarios, providing a practical solution for medical and microscopy imaging where clean ground-truth data is often unavailable.
Introduction
Most modern image denoising algorithms rely on the assumption that noise is independent and identically distributed (i.i.d.). However, in real-world applications—particularly in medical imaging (CT, ultrasound) or microscopy—noise is frequently structured, appearing as horizontal streaks, speckles, or grid-like patterns.
Existing zero-shot methods, which learn to denoise using only the noisy image itself (e.g., Noise2Void or Noise2Fast), often fail in these cases. Because structured noise is spatially correlated, these models mistake the noise patterns for actual image features, inadvertently "learning" to preserve the noise rather than remove it. The challenge is to find a way to separate the underlying clean signal from these organized noise artifacts without having a clean reference image.
Methodology
The core of M2M is a clever sampling strategy that transforms a single noisy image into pairs of "de-structured" noisy images suitable for Noise2Noise (N2N) training.
The M2M Pipeline
- Block-wise Sampling: The image is partitioned into $3 \times 3$ non-overlapping patches.
- Directional Interpolation: For a target pixel, the algorithm calculates several estimates of its value by looking at its neighbors along different axes (horizontal, vertical, and diagonal).
- Generalized Median Filtration: Structured noise usually corrupts specific directions more than others. By taking the set of directional estimates and applying a median filter, the algorithm discards the "outliers"—the estimates most heavily corrupted by the structured noise. This leverages the statistical robustness of the median over the mean.
- Randomized Assignment (RAS): To avoid systematic bias, M2M randomly assigns these filtered values into pairs ($x_1, x_2$).
- Parallel Training: M2M trains nine independent, lightweight 3-layer convolutional neural networks (CNNs) in parallel, each specializing in one of the nine positions within the $3 \times 3$ patches.
- Inference: During the final step, the outputs of these nine networks are reassembled and averaged to produce the final clean image.
Key Results
The authors evaluated M2M using the Kodak dataset and fluorescence microscopy data, simulating realistic biomedical imaging conditions where structured noise is a common hurdle.
Performance and Efficiency
- Superiority over Baselines: In scenarios with high noise correlation lengths, M2M achieved PSNR gains of 3-9 dB over other zero-shot methods like Noise2Fast and Zero-Shot Noise2Noise.
- Beating Non-Blind Methods: Surprisingly, M2M often outperformed BM3D, a classic gold-standard denoising algorithm, even when BM3D was provided with the exact noise variance as an input.
- Efficiency: Despite training nine networks, the process is highly efficient. Denoising a standard image takes roughly tens of seconds, making it practical for real-world laboratory settings.
Conclusion
Median2Median proves that complex, structured noise can be suppressed effectively without massive external datasets. By leveraging directional sampling and median filtration, M2M "breaks" the spatial correlation of noise, making it appear i.i.d. to the neural network. Future work may extend this framework to 3D volumetric data or video, providing even more directional information for the median filter to exploit, further improving edge preservation and noise suppression.
Summary updated