Boundary-Preserved Deep Denoising of Stochastic Resonance Enhanced Multiphoton Images

Sheng-Yong Niu, Lun‐Zhang Guo, Yue Li, Zhiming Zhang, Tzung‐Dau Wang, Kai-Chun Liu, You-Jin Li, Yu Tsao, Tzu‐Ming LiuIEEE Journal of Translational Engineering in Health and Medicine 2022 · DOI

TL;DR

This paper introduces a novel deep learning approach, a fully convolutional denoising autoencoder (DDAE), to significantly improve the quality of low-photon budget multiphoton microscopy images. By preserving crucial cellular boundaries while effectively removing noise, this method promises to enhance the accuracy of downstream analyses like segmentation and texture analysis, ultimately supporting clinical evaluation.

What Problem Does This Paper Address?

High-speed, deep-tissue imaging is becoming increasingly vital in biomedical research, enabling scientists to observe cellular interactions in complex environments. However, acquiring these images, especially with techniques like three-photon fluorescence (3PF) and third-harmonic generation (THG) microscopy, often involves low photon budgets. This leads to images with a poor signal-to-noise ratio (SNR), where signals are comparable to or even masked by background noise. This "stochastic resonance" phenomenon, where noise can paradoxically help detect weak signals, makes traditional denoising methods problematic. These methods often suppress noise but also blur or remove important cellular features, particularly the boundaries of nuclei and cells, which are critical for subsequent analysis like segmentation and texture mapping. Therefore, there's a pressing need for a denoising method that can effectively reduce noise without sacrificing these vital morphological details.

Methods

The core of this work is a fully convolutional deep denoising autoencoder (DDAE). This neural network architecture is designed to take a noisy image as input and reconstruct a clean, denoised version. The autoencoder consists of an encoder that progressively downsamples the image, capturing its essential features, and a decoder that upsamples these features to reconstruct the image. The "fully convolutional" aspect means that the network uses convolutional layers throughout, making it suitable for processing images of arbitrary sizes and preserving spatial information.

The authors trained their DDAE model using a dataset where the "ground truth" or low-noise answer was generated by averaging a large number of acquired images (200 images) of the same location. This averaging process effectively reduces noise while preserving the underlying signal. The DDAE was trained to minimize the mean-squared error between its output and these averaged images.

For comparison, the authors also evaluated traditional denoising methods:

  • Gaussian filter: A common low-pass filter that smooths images.
  • Median filter: A non-linear filter effective at removing salt-and-pepper noise.
  • BM3D (Block-matching and 3D filtering): A state-of-the-art traditional denoising algorithm known for its effectiveness in various image denoising tasks.

Datasets

The study utilized RAW 264.7 murine macrophage cell lines for imaging.

  • 3PF imaging: Cells were stained with Hoechst 33342 for nuclei visualization.
  • THG imaging: No labeling was used, as lipid granules in M1-activated macrophages produce THG signals.

For training and evaluation, approximately 200 images per location were acquired. The "low-noise answer" for DDAE training was generated by averaging these 200 images. The authors performed 5-fold cross-validation on 31 locations for each imaging modality (3PF and THG).

Key Insights and Ideas

  • Stochastic Resonance is Tricky: The paper highlights that in low-light microscopy, noise isn't just a simple additive disturbance; it can interact with weak signals in a way that makes them detectable. This "stochastic resonance" makes traditional filters, which assume simple noise models, less effective and prone to removing these subtle but important signals.
  • Deep Learning's Advantage: The DDAE's ability to learn complex, non-linear relationships within the data allows it to distinguish between true signals (even weak ones) and noise, a task that linear filters struggle with.
  • Boundary Preservation is Key: The primary innovation is the DDAE's success in preserving the sharp boundaries of nuclei and cells, which is often a casualty of aggressive denoising. This is crucial for downstream analyses.
  • Trade-off Optimization: The DDAE offers a better balance between noise reduction (measured by PSNR) and structural preservation (measured by SSIM) compared to traditional methods, especially for challenging low-photon budget scenarios.

Key Results

  • Improved SNR and SSIM: The DDAE model achieved significantly better signal-to-noise ratio (PSNR) and structural similarity (SSIM) scores compared to Gaussian and median filters for both 3PF and THG images. For 3PF, the average PSNR was 28.86 and SSIM was 0.89. For THG, the average PSNR was 21.66 and SSIM was 0.70.
  • Superior Boundary Preservation: The DDAE demonstrated superior preservation of nuclear and cellular boundaries, as quantified by F1-scores of 0.662 for 3PF and 0.736 for THG. This was visually confirmed by the retention of sharp contours in the denoised images.
  • BM3D Performance: While DDAE generally outperformed traditional methods, BM3D showed competitive PSNR for 3PF images and higher PSNR for THG images in some configurations, but often at the cost of boundary detail.
  • Efficiency: The DDAE model can process images in seconds, making it suitable for high-throughput applications.

Limitations and Future Work

  • False Negatives in 3PF: For 3PF images, the DDAE generated slightly more false negatives in nuclear boundary analysis compared to BM3D, although its overall F1-score was higher.
  • False Positives in THG: For THG images, the DDAE might produce more false positives and fewer false negatives in cytoplasmic boundary analysis.
  • Future Directions: The authors plan to explore more advanced deep learning models for even better denoising performance. They also aim to develop lightweight DDAE models for low-resource environments (e.g., using pruning and tinyML) and investigate training with less ideal datasets (e.g., without clean ground truth). The ultimate goal is to integrate this denoising capability into real-time microscopy analysis pipelines for applications like high-throughput segmentation and 3D morphodynamic analysis.

Summary updated

Abstract

The proposed deep denoising system can enhance the quality of microscopic images and effectively support clinical evaluation and assessment.
BibTeX
@article{niu2022boundary,
  author = {Sheng-Yong Niu and Lun‐Zhang Guo and Yue Li and Zhiming Zhang and Tzung‐Dau Wang and Kai-Chun Liu and You-Jin Li and Yu Tsao and Tzu‐Ming Liu},
  title = {{Boundary-Preserved Deep Denoising of Stochastic Resonance Enhanced Multiphoton Images}},
  journal = {IEEE Journal of Translational Engineering in Health and Medicine},
  year = {2022},
  doi = {10.1109/jtehm.2022.3206488},
}
Figure