Poisson2Sparse: Self-supervised Poisson Denoising from a Single Image

Calvin-Khang Ta, Abhishek Aich, Akash Gupta, Amit K. Roy-Chowdhury

TL;DR

Poisson2Sparse is a self-supervised framework designed to denoise biomedical images corrupted by signal-dependent Poisson noise using only a single noisy input. By "unrolling" a classic sparsity-enforcing optimization algorithm into a recurrent neural network structure, the method recovers fine structural details without ever requiring clean ground-truth data for training.

What Problem Does This Paper Address?

Most image denoising research assumes "Additive White Gaussian Noise" (AWGN), where the noise level is constant regardless of the image brightness. However, in biomedical imaging—like fluorescence microscopy or X-rays—the noise is physically tied to the number of photons hitting the sensor. This results in Poisson noise, which is signal-dependent: brighter areas are noisier.

Furthermore, deep learning models usually require thousands of "clean/noisy" image pairs to learn how to denoise. In clinical or research settings, obtaining a "clean" version of a cell or a beating heart is often impossible or requires long exposure times that damage the biological sample. The authors address this by proposing a method that learns to denoise using only the single noisy image provided at test time (a concept known as "internal learning").

Methods

The authors combine classical signal processing theory with modern deep learning through three key components:

  1. Unrolled Optimization (ISTA): The researchers take the Iterative Shrinkage Thresholding Algorithm (ISTA)—a traditional method for solving sparse reconstruction problems—and approximate its steps using a Recurrent Neural Network (RNN). Instead of a fixed mathematical operator, they use a learnable Encoder-Decoder structure to find a "sparse representation" of the image.
  2. Convolutional Sparse Coding (CSC): To ensure the model works regardless of the input image size, they use convolutions to learn a "dictionary" of image patches. This allows the network to recognize repetitive structures (like cell membranes or vessel walls) across the image.
  3. Self-Supervised Training: To train without ground truth, they use a technique called Random Neighbor Down-sampling. They split the single noisy image into two sub-sampled versions. Because adjacent pixels in a real image are highly correlated but their noise realizations are independent, the network can be trained to predict the underlying signal of one sub-sample from the other.
  4. Poisson-Specific Loss: Unlike standard models that use Mean Squared Error (which assumes Gaussian noise), this paper derives a loss function based on the maximum log-likelihood of a Poisson process, making it mathematically tailored to the physics of the sensors.

Datasets

The authors evaluated their model on two primary datasets:

  • Fluorescence Microscopy Denoising (FMD) Dataset: Contains 12,000 real-world microscopy images. Clean "ground truth" is only available here by averaging 50 noisy frames, which the authors use solely for evaluation.
  • PINCAT Dataset: A simulated cardiac perfusion MRI dataset. The authors manually added Poisson noise at varying intensities ($\lambda = 10, 20, 40$) to test the model's robustness against different noise levels.

Key Insights and Ideas

  • Sparsity as a Filter: The core "aha" moment is that noise is inherently non-sparse and random, while the underlying biological structure is repetitive and can be represented by a few "dictionary" elements. By forcing the network to find a sparse representation, the noise is naturally filtered out as "spurious components."
  • Architecture as a Prior: Similar to "Deep Image Prior," the structure of the network itself acts as a regularizer. The network is physically "reluctant" to fit high-frequency random noise, preferring to fit the spatially correlated structures of the actual image.
  • Physics-Informed Deep Learning: By replacing the standard MSE loss with a Poisson-derived loss, the model achieves significantly better results on biomedical data where the Gaussian assumption fails.

Key Results

  • Quantitative Superiority: On the PINCAT dataset, Poisson2Sparse outperformed state-of-the-art methods like BM3D (traditional) and DIP/Self2Self (self-supervised) by roughly 2 dB in PSNR (Peak Signal-to-Noise Ratio) on average.
  • Detail Recovery: Qualitatively, the method was much better at preserving "subtle details" like the thin lines of a rib cage or the fine textures in a cell, which other methods tended to over-smooth into blurry blobs.
  • Stability: The authors noted that competing self-supervised methods like Self2Self were occasionally unstable, sometimes producing completely blank images, whereas Poisson2Sparse remained robust across the dataset.

Limitations and Future Work

  • Mixed Noise Models: While the paper focuses on pure Poisson noise, real-world sensors often have a mix of Poisson (shot) and Gaussian (read) noise. The authors noted that on the FMD dataset—where the noise isn't "pure" Poisson—their SSIM (Structural Similarity) scores were slightly lower than some competitors, even though visual quality was high.
  • Computational Cost: Because the model trains on the specific image it is denoising, it requires several minutes of GPU time per image (about 7 minutes for a 128x128 image). This makes it less suitable for real-time applications compared to pre-trained supervised models.
  • Future Directions: The authors suggest that extending this framework to handle "Poisson-Gaussian" mixtures would make it even more practical for a wider range of commercial imaging hardware.

Summary updated

Abstract

Image enhancement approaches often assume that the noise is signal independent, and approximate the degradation model as zero-mean additive Gaussian. However, this assumption does not hold for biomedical imaging systems where sensor-based sources of noise are proportional to signal strengths, and the noise is better represented as a Poisson process. In this work, we explore a sparsity and dictionary learning-based approach and present a novel self-supervised learning method for single-image denoising where the noise is approximated as a Poisson process, requiring no clean ground-truth data. Specifically, we approximate traditional iterative optimization algorithms for image denoising with a recurrent neural network that enforces sparsity with respect to the weights of the network. Since the sparse representations are based on the underlying image, it is able to suppress the spurious components (noise) in the image patches, thereby introducing implicit regularization for denoising tasks through the network structure. Experiments on two bio-imaging datasets demonstrate that our method outperforms the state-of-the-art approaches in terms of PSNR and SSIM. Our qualitative results demonstrate that, in addition to higher performance on standard quantitative metrics, we are able to recover much more subtle details than other compared approaches. Our code is made publicly available at https://github.com/tacalvin/Poisson2Sparse .
BibTeX
@inproceedings{ta2022poisson,
  author = {Calvin-Khang Ta and Abhishek Aich and Akash Gupta and Amit K. Roy-Chowdhury},
  title = {{Poisson2Sparse: Self-supervised Poisson Denoising from a Single Image}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2022},
  url = {https://openreview.net/forum?id=K-nW-KVo9S},
}

Loading related content...

Figure