DISCo: Deep Learning, Instance Segmentation, and Correlations for Cell Segmentation in Calcium Imaging

Elke Kirschbaum, Alberto Bailoni, Fred A. Hamprecht

Abstract

Automatically segmenting individual cells from noisy, dynamic calcium imaging videos is a fundamental challenge in neuroscience. We present DISCo (Deep learning, Instance Segmentation, and Correlations), a robust pipeline that integrates segment-wise temporal pixel correlations with spatial summary images. By utilizing a two-stage neural network architecture followed by signed graph partitioning, DISCo effectively identifies both active and inactive cells. Our method achieves state-of-the-art results on the Neurofinder benchmark, demonstrating the power of combining spatial and temporal information through learned aggregation.

Introduction

Calcium imaging allows researchers to observe the activity of hundreds of neurons simultaneously in vivo. However, extracting precise cell boundaries (instance segmentation) is difficult due to overlapping cells, low signal-to-noise ratios, and the presence of inactive cells. Existing methods often rely exclusively on either spatial morphology or temporal activity, leading to missed cells or poor boundary definition. DISCo addresses these limitations by leveraging both contexts in a computationally efficient manner.

Methods

The DISCo pipeline consists of three primary stages: temporal feature extraction, neural network transformation, and instance-level partitioning.

Segment-wise Correlations

To preserve sparse signals that might be lost when averaging over an entire video, DISCo divides the temporal sequence into 10 segments ($n=1 \dots 10$). Pearson correlations are calculated locally for each pixel relative to its 15 nearest neighbors within these segments. This approach significantly boosts the signal-to-noise ratio for neurons with infrequent firing patterns by focusing on shorter windows of activity.

Visual representation of segment-wise correlations ($c^n$) calculated for two pixel time-series ($x_{pq}$ and $x_{p'q'}$).

Neural Network Architecture and Training

The architecture employs a two-stage approach. First, a 3D CNN (CNN 1) aggregates the segment-wise correlations, learning complex temporal dynamics that outperform simple statistical pooling. The resulting features are concatenated with a mean intensity projection (summary image) and processed by a 2D U-Net (CNN 2). This stage predicts pixel-to-pixel affinities—indicating the likelihood that adjacent pixels belong to the same cell—and a foreground/background mask.

The DISCo pipeline overview, showing the flow from input time-signals to the final output segmentation via two CNN stages and graph partitioning.

Graph Partitioning

Finally, the predicted affinities are used as edge weights in an undirected grid graph. After pruning background pixels using the predicted mask, the Generalized Algorithm for Signed graph Partitioning (GASP) clusters the remaining pixels into distinct cell instances. Unlike traditional watershed methods, GASP does not require predefined seeds or cluster counts.

Results

DISCo was evaluated on the Neurofinder public benchmark, which includes 19 training and 9 test videos across five diverse series (00-04).

  • Benchmark Performance: A single DISCo model achieved an average F1-score of 0.63. When specialized models (DISCos) were trained for each dataset series, the average F1-score rose to 0.67, topping the Neurofinder leaderboard.
  • Ablation Study: Combining spatial and temporal data proved critical. The hybrid model (0.63 F1) significantly outperformed models using only correlations (0.50 F1) or only summary images (0.49 F1).
  • Generalization: The model demonstrated strong few-shot generalization, achieving an F1-score of 0.58 even when trained on only a single video from a series.

Conclusion

DISCo provides a powerful framework for cell segmentation by effectively combining spatial and temporal cues. While performance bottlenecks vary across datasets—with some limited by recall and others by precision—the learned aggregation of temporal segments and direct instance segmentation via graph partitioning represent significant advances in the field.

Summary updated

Abstract

Calcium imaging is one of the most important tools in neurophysiology as it enables the observation of neuronal activity for hundreds of cells in parallel and at single-cell resolution. In order to use the data gained with calcium imaging, it is necessary to extract individual cells and their activity from the recordings. We present DISCo, a novel approach for the cell segmentation in calcium imaging videos. We use temporal information from the recordings in a computationally efficient way by computing correlations between pixels and combine it with shape-based information to identify active as well as non-active cells. We first learn to predict whether two pixels belong to the same cell; this information is summarized in an undirected, edge-weighted graph which we then partition. Evaluating our method on the Neurofinder public benchmark shows that DISCo outperforms all existing models trained on these datasets.
BibTeX
@inproceedings{kirschbaum2020disco,
  author = {Elke Kirschbaum and Alberto Bailoni and Fred A. Hamprecht},
  title = {{DISCo: Deep Learning, Instance Segmentation, and Correlations for Cell Segmentation in Calcium Imaging}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  doi = {10.1007/978-3-030-59722-1\_15},
}
Figure