Deep-Learning Recognition of Scanning Transmission Electron Microscopy: Quantifying and Mitigating the Influence of Gaussian Noises

Hanlei Zhang, Jincheng Bai, Xiabo Chen, Can Li, Chuanjian Zhong, Jiye Fang, Guangwen ZhouarXiv 2024 · DOI · arXiv

Introduction

Scanning Transmission Electron Microscopy (STEM) has reached a point where it can capture atomic-level details at millisecond speeds. However, this "data deluge" creates a massive bottleneck: human researchers cannot manually label and measure thousands of nanoparticles in a reasonable timeframe. While computer vision seems like a natural fit for automation, electron microscopy images are notoriously noisy. Gaussian noise—arising from electronics and the stochastic nature of electron beams—can easily confuse standard algorithms. This study seeks to quantify exactly when deep learning fails due to noise and how to mitigate these effects using advanced denoising techniques.

Methodology and Framework

The researchers developed a framework comparing traditional "non-smart" thresholding methods against a deep-learning approach using Mask R-CNN (via the Detectron2 platform). Mask R-CNN is an instance segmentation neural network that identifies individual objects and provides a precise pixel-level mask for each.

To rigorously test the limits of the model, the team used simulated datasets mimicking realistic conditions, including nanoparticles submerged in liquid layers with varying levels of artificial Gaussian noise. This allowed for a direct comparison of how noise, particle shape, and particle size influence recognition accuracy.

Conceptual framework comparing traditional thresholding with machine learning approaches for nanoparticle recognition across different variables.

Quantifying the Impact of Noise

A key finding of the study is the "smartness gap" between traditional methods and deep learning. Traditional thresholding (such as Otsu's method) fails under noise because it only evaluates local pixel intensity, leading to "broken" masks and internal vacancies. Mask R-CNN, by contrast, understands the semantic shape of a particle, allowing it to ignore local fluctuations.

However, even deep learning has its limits. The study identified a critical Signal-to-Noise Ratio (SNR) threshold. At an SNR of approximately 7.5, the Mask R-CNN model became "unavailable," failing to detect particles entirely. This provides a concrete benchmark for when preprocessing becomes mandatory in the microscopy pipeline.

Comparison of Mask R-CNN and semantic segmentation performance across varying noise levels, highlighting the failure point at SNR = 7.4870.

Geometric Factors and Mitigation Strategies

Beyond noise intensity, the study introduced the Boundary-to-Area Ratio ($R_{ba}$) as a critical metric for AI performance. Particles with high $R_{ba}$, such as thin nanorods, are significantly harder to recognize than spheres. Because noise primarily degrades boundaries, a high edge-to-volume ratio makes the particle's signal indistinguishable from the background.

To overcome these limitations, the researchers tested various denoising strategies. They found that Non-Local Means (NLM) filters were superior to standard Gaussian blurs. While Gaussian blurs smudge edges, NLM preserves sharp boundaries by looking for similar patterns across the entire image. Applying an NLM filter allowed the model to achieve 96.63% recognition accuracy on high-noise images where it had previously failed.

Conclusion

This research demonstrates that while Gaussian noise and complex particle geometries pose significant challenges to automated microscopy, they can be quantified and mitigated. By integrating NLM denoising with Mask R-CNN, researchers can restore high-accuracy recognition even in low-signal environments, paving the way for fully automated, high-throughput electron microscopy analysis.

Summary updated

BibTeX
@misc{zhang2024deep,
  author = {Hanlei Zhang and Jincheng Bai and Xiabo Chen and Can Li and Chuanjian Zhong and Jiye Fang and Guangwen Zhou},
  title = {{Deep-Learning Recognition of Scanning Transmission Electron Microscopy: Quantifying and Mitigating the Influence of Gaussian Noises}},
  year = {2024},
  eprint = {2409.16637},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.16637},
}
Figure