Fluctuation-Based Deconvolution in Fluorescence Microscopy Using Plug-and-Play Denoisers

Vasiliki Stergiopoulou, Subhadip Mukherjee, Luca Calatroni, Laure Blanc-Féraud

Introduction

Fluorescence microscopy is a cornerstone of biological research, yet it is fundamentally constrained by the diffraction limit of light. Objects positioned closer than approximately 200 nm typically appear as a single blurred entity. While super-resolution techniques like Single Molecule Localization Microscopy (SMLM) can circumvent this limit, they often require high-intensity lasers or lengthy acquisition times, which can be detrimental to live biological samples.

A promising alternative involves analyzing temporal fluctuations—the stochastic "blinking" of fluorescent molecules over time. Existing methods such as SPARCOM and COL0RME utilize the statistical variance of these fluctuations to reconstruct sub-diffraction features. However, these approaches often rely on rigid "sparsity" priors, which assume images are composed of isolated points. When applied to continuous structures like the cellular cytoskeleton, these methods frequently produce "beads-on-a-string" artifacts. This paper introduces PnP-COL0RME, a framework that replaces traditional mathematical priors with deep learning-based Plug-and-Play (PnP) denoisers to achieve smoother, more biologically accurate reconstructions.

Methodology: The PnP-COL0RME Framework

The researchers propose a modular approach to image restoration by balancing data fidelity with a learned image prior. Instead of a hand-crafted mathematical formula, the "real image" characteristics are captured by a pre-trained deep neural network, specifically DRUNet, which has been optimized to recognize and denoise filament-like structures.

Covariance-Based Modeling

Rather than processing individual frames, the algorithm operates on the auto-covariance (variance) of a stack of 500 frames. This approach exploits the independent blinking of molecules to sharpen the final image. While the noise in covariance data follows a complex $\chi^2$ distribution, the authors demonstrate that with a sufficient number of frames ($T=500$), it can be effectively approximated as Gaussian noise, enabling the use of high-performance standard denoisers.

Two-Step Optimization

The reconstruction follows a two-step process:

  1. Support Estimation: The PnP denoiser estimates the spatial locations (support) of the fluorescent molecules.
  2. Intensity Estimation: A standard optimization step calculates the intensity of the molecules at the identified locations.

To ensure reliability, the authors utilized a specific "gradient-step" denoiser formulation. Unlike many "black box" AI applications, this ensures that the iterative algorithm mathematically converges to a stable, predictable solution.

Experimental Setup and Insights

The performance of PnP-COL0RME was evaluated using both simulated and real-world datasets:

  • Simulated Data: Sequences of 500 frames were generated based on the MT0 microtubule training dataset from the SMLM 2016 challenge to mimic fluctuating molecules.
  • Real Data: High-density SMLM acquisitions from the 2013 SMLM challenge, featuring complex, overlapping cellular structures, were used to test the model's robustness.

A key insight of this work is the decoupling of the physical microscope model from the image prior. Because the PnP approach is modular, the microscope's forward operator can be adjusted for different laboratory setups without requiring the neural network to be retrained. Furthermore, the study highlights that neural networks can "learn" the complex geometry of filaments—promoting continuity and curvature—far more effectively than traditional sparsity-based descriptions.

Results and Discussion

Quantitative analysis on simulated data showed that PnP-COL0RME achieved a Jaccard Index (JI) of 0.67 for location accuracy and a PSNR of 26.01 for intensity reconstruction. Qualitatively, the method significantly outperformed the previous state-of-the-art (COL0RME). By avoiding the punctuated artifacts of sparsity-based methods, PnP-COL0RME produced connected and realistic filaments, particularly evident in the 2013 SMLM challenge data where it successfully traced long, thin structures that appeared fragmented in other reconstructions.

Limitations and Future Directions

Despite its success, the authors noted several areas for further development:

  • Model Complexity: The current iteration only utilizes auto-covariance (the diagonal of the covariance matrix). Incorporating cross-pixel fluctuations (cross-terms) could further enhance accuracy.
  • Resolution Scaling: The model currently assumes the output grid matches the input. Integrating a downsampling operator in future versions could enable even higher super-resolution.
  • Data Diversity: In extremely challenging scenarios, the model occasionally produced incorrect filament duplications. This suggests that future training datasets must be more diverse and heterogeneous to account for a wider range of biological edge cases.

By bridging the gap between statistical physics and deep learning, PnP-COL0RME offers a flexible and powerful tool for high-fidelity fluorescence microscopy.

Summary updated

Abstract

The spatial resolution of images of living samples obtained by fluorescence microscopes is physically limited due to the diffraction of visible light, which makes the study of entities of size less than the diffraction barrier (around 200 nm in the x-y plane) very challenging. To overcome this limitation, several deconvolution and super-resolution techniques have been proposed. Within the framework of inverse problems, modern approaches in fluorescence microscopy reconstruct a super-resolved image from a temporal stack of frames by carefully designing suitable hand-crafted sparsity-promoting regularisers. Numerically, such approaches are solved by proximal gradient-based iterative schemes. Aiming at obtaining a reconstruction more adapted to sample geometries (e.g. thin filaments), we adopt a plug-and-play denoising approach with convergence guarantees and replace the proximity operator associated with the explicit image regulariser with an image denoiser (i.e. a pre-trained network) which, upon appropriate training, mimics the action of an implicit prior. To account for the independence of the fluctuations between molecules, the model relies on second-order statistics. The denoiser is then trained on covariance images coming from data representing sequences of fluctuating fluorescent molecules with filament structure. The method is evaluated on both simulated and real fluorescence microscopy images, showing its ability to correctly reconstruct filament structures with high values of peak signal-to-noise ratio (PSNR).
BibTeX
@inproceedings{stergiopoulou2023fluctuation,
  author = {Vasiliki Stergiopoulou and Subhadip Mukherjee and Luca Calatroni and Laure Blanc-Féraud},
  title = {{Fluctuation-Based Deconvolution in Fluorescence Microscopy Using Plug-and-Play Denoisers}},
  booktitle = {SSVM},
  year = {2023},
  doi = {10.1007/978-3-031-31975-4\_38},
}
Figure