Abstract
Automatically segmenting individual cells from noisy, dynamic calcium imaging videos is a fundamental challenge in neuroscience. We present DISCo (Deep learning, Instance Segmentation, and Correlations), a robust pipeline that integrates segment-wise temporal pixel correlations with spatial summary images. By utilizing a two-stage neural network architecture followed by signed graph partitioning, DISCo effectively identifies both active and inactive cells. Our method achieves state-of-the-art results on the Neurofinder benchmark, demonstrating the power of combining spatial and temporal information through learned aggregation.
Introduction
Calcium imaging allows researchers to observe the activity of hundreds of neurons simultaneously in vivo. However, extracting precise cell boundaries (instance segmentation) is difficult due to overlapping cells, low signal-to-noise ratios, and the presence of inactive cells. Existing methods often rely exclusively on either spatial morphology or temporal activity, leading to missed cells or poor boundary definition. DISCo addresses these limitations by leveraging both contexts in a computationally efficient manner.
Methods
The DISCo pipeline consists of three primary stages: temporal feature extraction, neural network transformation, and instance-level partitioning.
Segment-wise Correlations
To preserve sparse signals that might be lost when averaging over an entire video, DISCo divides the temporal sequence into 10 segments ($n=1 \dots 10$). Pearson correlations are calculated locally for each pixel relative to its 15 nearest neighbors within these segments. This approach significantly boosts the signal-to-noise ratio for neurons with infrequent firing patterns by focusing on shorter windows of activity.
Neural Network Architecture and Training
The architecture employs a two-stage approach. First, a 3D CNN (CNN 1) aggregates the segment-wise correlations, learning complex temporal dynamics that outperform simple statistical pooling. The resulting features are concatenated with a mean intensity projection (summary image) and processed by a 2D U-Net (CNN 2). This stage predicts pixel-to-pixel affinities—indicating the likelihood that adjacent pixels belong to the same cell—and a foreground/background mask.
Graph Partitioning
Finally, the predicted affinities are used as edge weights in an undirected grid graph. After pruning background pixels using the predicted mask, the Generalized Algorithm for Signed graph Partitioning (GASP) clusters the remaining pixels into distinct cell instances. Unlike traditional watershed methods, GASP does not require predefined seeds or cluster counts.
Results
DISCo was evaluated on the Neurofinder public benchmark, which includes 19 training and 9 test videos across five diverse series (00-04).
- Benchmark Performance: A single DISCo model achieved an average F1-score of 0.63. When specialized models (DISCos) were trained for each dataset series, the average F1-score rose to 0.67, topping the Neurofinder leaderboard.
- Ablation Study: Combining spatial and temporal data proved critical. The hybrid model (0.63 F1) significantly outperformed models using only correlations (0.50 F1) or only summary images (0.49 F1).
- Generalization: The model demonstrated strong few-shot generalization, achieving an F1-score of 0.58 even when trained on only a single video from a series.
Conclusion
DISCo provides a powerful framework for cell segmentation by effectively combining spatial and temporal cues. While performance bottlenecks vary across datasets—with some limited by recall and others by precision—the learned aggregation of temporal segments and direct instance segmentation via graph partitioning represent significant advances in the field.
Summary updated