A comparison of high resolution image reconstruction methods from confocal array detector with a new deconvolution algorithm

Sylvain Prigent, Stéphanie Dutertre, Aurélien Bidaud-Meynard, Giulia Bertolin, Grégoire Michaux, Charles KervrannbioRxiv 2021 · DOI

TL;DR

This paper introduces a novel deconvolution algorithm (AD-S) for confocal microscopy using array detectors. AD-S outperforms existing methods by simultaneously merging signals from multiple sub-detectors and deconvolving them, leading to significantly improved resolution, contrast, and homogeneity in reconstructed images.

What Problem Does This Paper Address?

Confocal microscopy, while powerful, is limited by optical resolution. Array detectors, which use multiple sub-detectors to capture slightly different views of the sample, offer a way to increase resolution and light collection. However, reconstructing a high-resolution image from these multiple sub-detector signals is challenging. Existing methods fall into two categories: sub-detector combination methods that offer either improved contrast or resolution, but not both, and deconvolution-based methods that can achieve both but may introduce artifacts. This paper aims to bridge this gap by proposing a new deconvolution approach that effectively leverages the full information from the array detector to achieve superior reconstruction quality.

Methods

The authors compare several existing reconstruction methods, including:

  • Pseudo-confocal (PC): Simple summation of sub-detector images.
  • Image Scanning Microscopy (ISM): Spatially reassigns signals from sub-detectors to the central detector.
  • Fluorescence Emission Difference (FED) and Image Scanning FED (ISFED): Methods that subtract signals from "outer" rings of detectors from "inner" rings to enhance resolution. These methods rely on a critical subtraction factor ($\epsilon$) that needs careful calibration.
  • Deconvolution-based methods applied to ISM images: Richardson-Lucy (ISM-RL) and Wiener (ISM-W) filters applied to the ISM image.
  • ISM-S: A method using the SPITFIR(e) algorithm on the ISM image.

The core contribution is the AD-S (Array Detector - SPITFIR(e)) algorithm. This novel approach directly processes the raw array detector data. It modifies the SPITFIR(e) algorithm's energy function to incorporate weights for different groups of sub-detectors, giving more importance to outer detectors (higher frequencies) than inner ones. This allows for simultaneous merging of multiple sub-detector images and deconvolution, overcoming the limitations of processing the already merged ISM image. The authors also discuss a data-driven method for estimating the optimal subtraction factor ($\epsilon$) for IFED and ISFED methods using a local Stein's Unbiased Risk Estimate (SURE).

Datasets

The paper evaluates the performance of the reconstruction methods using:

  • Calibration slides: Specifically, a "donut" shaped object and a vertical line pattern from an Argolight calibration slide. These provide controlled structures for assessing resolution and contrast.
  • Real biological samples:
    • MCF7 cells expressing mito-GFP (labeling mitochondria).
  • C. elegans intestine expressing ERM-1::mNeonGreen fusion protein (labeling intestinal microvilli).

The C. elegans sample was used to evaluate performance under varying signal-to-noise ratios (SNR) by capturing images at different pixel dwell times (acquisition speeds).

Key Insights and Ideas

  • Simultaneous merging and deconvolution is superior: The AD-S algorithm's ability to perform both tasks concurrently on the raw sub-detector data is a key advantage over methods that first merge signals (like ISM) and then deconvolve. This preserves more high-frequency information.
  • Weighting outer detectors is crucial: Giving more importance to signals from outer sub-detectors in the AD-S algorithm allows for better capture of fine details and higher resolution.
  • Deconvolution methods offer better resolution than linear methods: While linear methods like PC and ISM provide good SNR, deconvolution-based approaches (ISM-RL, ISM-W, ISM-S, and AD-S) consistently achieve higher resolution.
  • AD-S balances resolution and robustness: AD-S not only provides high resolution but also demonstrates better robustness to noise and produces more homogeneous intensity distributions compared to other deconvolution methods.
  • Adaptive $\epsilon$ estimation is important: The paper highlights that the optimal subtraction factor ($\epsilon$) for IFED/ISFED methods is not fixed and should be adaptively adjusted based on SNR, suggesting the utility of data-driven estimation techniques.

Key Results

  • Donut object: AD-S produced the thinnest and most homogeneous ring profile, with an average thickness of 252 nm, outperforming other methods.
  • Vertical line pattern: AD-S and other deconvolution methods could distinguish lines down to 250 nm, while linear methods struggled beyond 350 nm.
  • Real data (MCF7 cells): AD-S provided the best resolution and was more robust to noise than competing deconvolution methods.
  • Real data (C. elegans microvilli): AD-S and deconvolution methods better enhanced the 100 nm large microvilli. AD-S showed superior background removal.
  • SNR robustness (C. elegans): AD-S consistently provided the best contrast as SNR decreased (i.e., as acquisition speed increased), outperforming linear methods significantly.
  • Computational performance: Linear methods (PC, ISM, IFED, ISFED) are faster (under 1 second per image), while deconvolution methods are about twice as slow (around 2 seconds per image on a core i7 CPU for a 512x512 image).

Limitations and Future Work

  • PSF modeling: The performance of AD-S relies on the PSF being approximately modeled by a Gaussian model. Deviations from this assumption could impact results.
  • Sub-detector co-registration: The paper notes that satisfying co-registration of sub-detectors is important for optimal performance.
  • 2D vs. 3D: While the comparison was performed on 2D images, all tested methods have been implemented for 3D image processing, suggesting future work could focus on 3D reconstruction performance.
  • Computational cost: While AD-S is superior, its computational cost is higher than linear methods, which might be a consideration for real-time applications. Future work could explore optimizations for speed.

Summary updated

Abstract

Array detector allows a resolution gain for confocal microscopy by combining images sensed by a set of photomultipliers tubes (or sub-detectors). Several methods have been proposed to reconstruct a high resolution image by linearly combining sub-detector images. To overcome the limitations of these techniques, we propose a new reconstruction method that takes the full stack of spatially reassigned detector signals as input. We show on both calibration slides and real data that our deconvolution method allows to achieve a better reconstruction performance in terms of resolution, image contrast, and spatial intensity homogeneity. The tested algorithms are available in an open source software.
BibTeX
@misc{prigent2021comparison,
  author = {Sylvain Prigent and Stéphanie Dutertre and Aurélien Bidaud-Meynard and Giulia Bertolin and Grégoire Michaux and Charles Kervrann},
  title = {{A comparison of high resolution image reconstruction methods from confocal array detector with a new deconvolution algorithm}},
  year = {2021},
  doi = {10.1101/2021.08.02.454749},
  howpublished = {bioRxiv},
}
Figure