DISCo for the CIA: Deep learning, Instance Segmentation, and Correlations for Calcium Imaging Analysis

Elke Kirschbaum, Alberto Bailoni, Fred A. Hamprecht

TL;DR

DISCo introduces a highly efficient framework for cell segmentation in calcium imaging by combining segment-wise pixel correlations with deep learning and graph partitioning. By blending temporal dynamics with spatial shape information, it accurately identifies both active and inactive neurons. This approach achieves state-of-the-art performance on the Neurofinder benchmark without the heavy computational toll of full 3D convolutional networks.

What Problem Does This Paper Address?

Calcium imaging is a vital neurophysiology technique that allows researchers to observe the activity of large populations of neurons simultaneously at single-cell resolution. However, converting these raw videos into usable data requires accurately isolating individual cell locations—a task known as cell segmentation.

Current methods fall into two flawed camps: deep learning models that rely on time-averaged "summary images" (which discard crucial temporal dynamics and fail when cells overlap), and purely correlation-based methods (which fail to detect inactive or weakly active neurons because their signals blend into background noise). Full 3D convolutional neural networks (CNNs) can process the entire video tensor but are computationally expensive and difficult to scale. DISCo bridges this gap by efficiently leveraging both temporal correlation and spatial shape information.

Methods

The DISCo (Deep learning, Instance Segmentation, and Correlations) workflow consists of three main stages:

  1. Segment-Wise Correlations: Instead of calculating pixel correlations across the entire video timeline, DISCo splits the video into $N=10$ temporal segments. For each pixel, it computes the Pearson correlation coefficient relative to 15 neighboring pixels within a small radius. This preserves fine-grained temporal dynamics and boosts the signal-to-noise ratio.
  2. Deep Learning Architecture: A small 3D CNN aggregates the segment-wise correlation stacks. This output is concatenated with a standard mean-intensity summary image and fed into a 2D U-Net. The U-Net outputs two predictions: pixel-to-pixel affinities (the probability that adjacent pixels belong to the same cell) and a foreground-background mask.
  3. Instance Segmentation via Graph Partitioning: The predicted affinities are converted into weights for an undirected signed graph. DISCo applies the Generalized Algorithm for Signed graph Partitioning (GASP) to cluster the pixels into distinct cell instances directly, bypassing the need for manual seed points or threshold tuning.
Figure 1: DISCo workflow. For five exemplary pixel signals from the Neurofinder training set we show how they are processed by DISCo. First, the signals are split into temporal segments and segment-wise correlations between the pixels are computed. Next, the information from the segments is aggregated by a first CNN. These aggregated correlations are complemented by a mean intensity projection over time for each pixel. This combination of temporal and spatial information is processed by a second CNN which outputs affinities between pixels and a foreground-background prediction. Both CNNs are trained end-to-end. The predicted affinities are used by a graph partitioning algorithm to gain the final instance segmentation.

Datasets

The authors evaluated DISCo on the Neurofinder public benchmark, a standard challenge consisting of 19 calcium imaging training videos with ground-truth cell annotations and 9 test videos with undisclosed ground truth. The dataset is split into five distinct series (00, 01, 02, 03, and 04), each recorded under different experimental conditions and featuring varying proportions of active versus inactive neurons.

Key Insights and Ideas

  • The Power of Chunking Time: Full-video correlations often wash out sparse neuronal spikes with background noise. Computing correlations over shorter segments dramatically boosts the signal-to-noise ratio, making transiently active cells stand out clearly against the background (as illustrated in Figure 3).
  • Hybrid Inputs for Hybrid States: Combining summary images (spatial/shape info) with correlations (temporal info) allows a single model to excel at finding both structurally visible inactive cells and structurally faint but highly active cells.
  • Learned Temporal Aggregation: Rather than using simple heuristics like the mean or max to compress temporal segments, using a 3D CNN to learn the aggregation function yields significantly better segmentation accuracy.

Key Results

DISCo achieved state-of-the-art results on the Neurofinder leaderboard:

  • Single-Model Performance: A single DISCo model trained across all datasets achieved an average F1-score of 0.63, outperforming other single-model baselines like UNet2DS (0.57) and pure HNCcorr (0.49).
  • Multi-Model Performance: When specialized networks were trained for each of the five dataset series individually (DISCos), the approach reached an average F1-score of 0.67, surpassing the previous top-performing 3dCNN method (0.66).
  • Ablation Success: The authors' lesion study confirmed that the combination of segment-wise correlations and summary images provides a substantial performance boost over using either input modality alone.

Limitations and Future Work

The authors note that DISCo's performance behavior is inconsistent across different dataset types. On some datasets, the model suffers from relatively low recall compared to its precision (e.g., series 00), while on others, precision is the limiting factor (e.g., series 01). The underlying cause of this discrepancy remains an open question, and future work will focus on analyzing dataset-specific properties to resolve these imbalances.

Summary updated

Abstract

Calcium imaging is one of the most important tools in neurophysiology as it enables the observation of neuronal activity for hundreds of cells in parallel and at single-cell resolution. In order to use the data gained with calcium imaging, it is necessary to extract individual cells and their activity from the recordings. We present DISCo, a novel approach for the cell segmentation in calcium imaging videos. We use temporal information from the recordings in a computationally efficient way by computing correlations between pixels and combine it with shape-based information to identify active as well as non-active cells. We first learn to predict whether two pixels belong to the same cell; this information is summarized in an undirected, edge-weighted grid graph which we then partition. In so doing, we approximately solve the NP-hard correlation clustering problem with a recently proposed greedy algorithm. Evaluating our method on the Neurofinder public benchmark shows that DISCo outperforms all existing models trained on these datasets.
BibTeX
@inproceedings{kirschbaum2020disco,
  author = {Elke Kirschbaum and Alberto Bailoni and Fred A. Hamprecht},
  title = {{DISCo for the CIA: Deep learning, Instance Segmentation, and Correlations for Calcium Imaging Analysis}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  doi = {10.1007/978-3-030-59722-1\_15},
}

Loading related content...

Figure