Deep learning reduces data requirements and allows real-time measurements in Imaging Fluorescence Correlation Spectroscopy

Wai Hoh Tang, Shao Ren Sim, Daniel Aik, Ashwin V.s. Nelanuthala, Thamarailingam Athilingam, Adrian Röllin, Thorsten WohlandbioRxiv 2023 · DOI

TL;DR

This paper introduces two deep learning models, FCSNet and ImFCSNet, that significantly reduce the data requirements and accelerate the analysis of Imaging Fluorescence Correlation Spectroscopy (Imaging FCS) measurements. These models enable real-time, model-agnostic estimation of diffusion coefficients, making this powerful technique more accessible and efficient for studying molecular dynamics in biological systems.

What Problem Does This Paper Address?

Imaging Fluorescence Correlation Spectroscopy (Imaging FCS) is a valuable technique for understanding molecular behavior within live biological samples. However, its widespread adoption is hindered by several significant limitations:

  • Data Hunger: Traditional Imaging FCS requires a large number of frames (e.g., 50,000 frames at 1 ms resolution) to achieve accurate parameter estimates, leading to lengthy acquisition times.
  • Slow Data Evaluation: The sheer volume of data generated by Imaging FCS makes post-acquisition analysis time-consuming.
  • Model Dependency: The analysis often relies on fitting experimental data to specific theoretical models. When analytical models are unavailable or complex, the evaluation process becomes even slower and more challenging, requiring specialized expertise.

These limitations restrict the temporal resolution achievable and make real-time analysis difficult, hindering the study of fast biological processes or experiments where rapid feedback is crucial.

Methods

The authors developed two Convolutional Neural Networks (CNNs) based on the ResNet architecture, incorporating skip connections for improved training stability.

  • FCSNet: This model takes autocorrelation functions (ACFs) as input, which are calculated from each pixel of an image stack. It is designed to predict diffusion coefficients (D) for 2D and 3D samples. FCSNet requires approximately 50,000 frames for evaluation, similar to traditional methods, but offers faster processing.
  • ImFCSNet: This model operates directly on raw intensity traces from 3x3 pixel patches of an image stack. A key innovation is its ability to achieve accurate parameter estimation with significantly less data – up to 100 times fewer frames than traditional methods. ImFCSNet also demonstrates robustness against sample defocusing.

Both CNNs were trained on simulated data, making them model-agnostic, meaning they do not rely on specific theoretical models for fitting. This allows them to be applied to a wider range of experimental conditions and sample types. The models output the natural logarithm of the diffusion coefficient (log D).

Datasets

  • Training Data: The CNNs (FCSNet and ImFCSNet) were trained on simulated data. This included simulating random walks in 2D (TIRF) and 3D (Gaussian Beam) under various conditions, including different diffusion coefficients (D) and signal-to-noise ratios (SNR).
  • Evaluation Data: The models were evaluated on experimental data from:
  • Supported lipid bilayers (DOPC and POPC)
    • Live CHO-K1 cells expressing a fluorescent protein
    • Fluorescent polystyrene beads
    • Early drosophila embryos

The authors note that they used non-overlapping sets of frames for evaluation, with ImFCSNet using 2,500 frames and FCSNet/NLS using 50,000 frames.

Key Insights and Ideas

  • Data Reduction is Key: The most striking insight is ImFCSNet's ability to achieve comparable results to traditional methods using orders of magnitude less data. This dramatically shortens acquisition times, opening doors to studying faster dynamics.
  • Model-Agnostic Analysis: Training on simulated data allows the CNNs to bypass the need for specific theoretical models, making them more versatile and applicable to complex or novel experimental setups where analytical solutions are scarce.
  • Robustness to Defocusing: ImFCSNet's remarkable ability to maintain stable diffusion coefficient estimates even when the sample is out of focus is a significant advantage, simplifying experimental procedures and improving data reliability.
  • Validation of Models: The CNNs' predictions, particularly when compared to established numerical models for SPIM-FCS, can help validate and refine theoretical approaches by providing independent, data-driven estimates.

Key Results

  • Speed-up: The CNNs offer substantial speed improvements in data evaluation. ImFCSNet, in particular, reduces evaluation times by a factor of 100-1,000, enabling near real-time analysis. FCSNet also provides significant speed-ups compared to NLS.
  • Data Requirement Reduction: ImFCSNet can achieve accurate parameter estimates using up to 20 times less data than traditional methods.
  • Accuracy: FCSNet demonstrates accuracy comparable to or better than NLS fitting, particularly in 2D and 3D measurements. ImFCSNet also provides good estimates, especially when averaged, and is more robust to defocusing.
  • Defocusing Robustness: ImFCSNet's predictions remain stable and consistent even with moderate sample defocusing, unlike NLS and FCSNet.
  • Application Range: The models were successfully applied to diverse biological samples, including lipid bilayers, cells, and drosophila embryos, demonstrating their broad applicability.

Limitations and Future Work

  • Explainability: The authors acknowledge the "black box" nature of CNNs, noting the lack of direct explainability in how they arrive at their predictions. This is a general challenge in deep learning.
  • Spatial Resolution (ImFCSNet): ImFCSNet evaluates 3x3 pixel patches, which reduces spatial resolution compared to pixel-wise analysis by FCSNet or NLS. While overlap mode can mitigate this, neighboring pixel estimates become interdependent.
  • SNR Sensitivity (ImFCSNet): ImFCSNet's predictions can exhibit larger fluctuations at lower signal-to-noise ratios.
  • Complex Systems: The current models are trained on simulated free diffusion of a single component. Future work will focus on estimating more parameters, handling heterogeneous systems with multiple components, and modeling complex diffusion processes or active transport.
  • Failure Cases: The authors highlight that ImFCSNet can fail in challenging samples (e.g., drosophila embryos with low SNR and potential multi-component diffusion), underscoring the need for careful determination of experimental settings where CNNs can reliably operate. They suggest using multiple models in conjunction or performing NLS spot checks for quality control.
  • Uncertainty Quantification: The authors propose exploring Bayesian approaches to quantify the uncertainty of CNN predictions, similar to how NLS provides goodness-of-fit statistics.

Summary updated

Abstract

Abstract Imaging Fluorescence Correlation Spectroscopy (Imaging FCS) is a powerful tool to extract information on molecular mobilities, actions and interactions in live cells, tissues and organisms. Nevertheless, several limitations restrict its applicability. First, FCS is data hungry, requiring 50,000 frames at 1 ms time resolution to obtain accurate parameter estimates. Second, the data size makes evaluation slow. Thirdly, as FCS evaluation is model-dependent, data evaluation is significantly slowed unless analytic models are available. Here we introduce two convolutional neural networks (CNNs) – FCSNet and Im-FCSNet – for correlation and intensity trace analysis, respectively. FCSNet robustly predicts parameters in 2D and 3D live samples. ImFCSNet reduces the amount of data required for accurate parameter retrieval by at least one order of magnitude and makes correct estimates even in moderately defocused samples. Both CNNs are trained on simulated data, are model-agnostic, and allow autonomous, real-time evaluation of Imaging FCS measurements.
BibTeX
@misc{tang2023deep,
  author = {Wai Hoh Tang and Shao Ren Sim and Daniel Aik and Ashwin V.S. Nelanuthala and Thamarailingam Athilingam and Adrian Röllin and Thorsten Wohland},
  title = {{Deep learning reduces data requirements and allows real-time measurements in Imaging Fluorescence Correlation Spectroscopy}},
  year = {2023},
  doi = {10.1101/2023.08.07.552352},
  howpublished = {bioRxiv},
}
Figure