A novel deep learning-based 3D cell segmentation framework for future image-based disease detection

Andong Wang, Qi Zhang, Yang Han, Sean G. Megason, Sahand Hormoz, Kishore Mosaliganti, Jacqueline C. K. Lam, Victor O. K. LiScientific 2022 · DOI

TL;DR

This paper introduces 3DCellSeg, a novel two-stage deep learning framework for accurate and efficient 3D cell segmentation, particularly in challenging scenarios with densely packed cells. By employing a lightweight network, a specialized loss function, and a unique clustering algorithm, 3DCellSeg significantly reduces the need for extensive hyperparameter tuning and demonstrates robust performance across diverse cell types.

What Problem Does This Paper Address?

Accurately segmenting cells in 3D microscopy images is fundamental for understanding biological processes and diagnosing diseases. However, existing deep learning methods struggle with densely packed cells, often leading to inaccurate segmentation. Furthermore, these methods typically require extensive manual tuning of numerous hyperparameters for each new dataset, making them cumbersome and time-consuming to adapt. This paper addresses these limitations by proposing a more robust, efficient, and user-friendly 3D cell segmentation pipeline.

Methods

The 3DCellSeg framework is a two-stage pipeline:

  • Stage I: Semantic Segmentation (3DCellSegNet & 3DCellSeg Loss)
    • 3DCellSegNet: A lightweight, U-Net-like convolutional neural network (CNN) designed for efficient processing of 3D cell membrane images. Its architecture is tailored to the recurrent patterns found in these images, offering faster inference than larger models used for general medical imaging.
  • 3DCellSeg Loss: A novel loss function that builds upon the Dice Loss. It incorporates weight matrices to specifically penalize low-confidence predictions near cell membranes, effectively mitigating the problem of clumped cells that often adhere to each other after semantic segmentation. This loss function aims to better distinguish individual cell boundaries.
  • Stage II: Instance Segmentation (TASCAN)
    • TASCAN (Touching Area-based Spatial Clustering of Applications with Noise): An efficient clustering algorithm inspired by DBSCAN. It first pre-clusters voxels into "super voxels" using a watershed algorithm. Then, TASCAN merges these super voxels based on their touching area. This approach significantly reduces computational complexity compared to voxel-wise clustering and requires only a single hyperparameter: the minimum touching area between super voxels.

Datasets

The study utilized four publicly available 3D cell membrane image datasets for training and evaluation:

  • HMS: Zebrafish transgenic embryos (animal cells).
  • ATAS: Arabidopsis thaliana apical stem cells (plant cells).
  • LRP: Arabidopsis thaliana lateral root primordia (plant cells).
  • Ovules: Arabidopsis thaliana ovules (plant cells).

The paper does not introduce new benchmarks but evaluates its method against existing ones on these established datasets.

Key Insights and Ideas

  • Domain-Specific Design Matters: By recognizing that cell membrane images have distinct characteristics compared to general medical images (like CT or MRI), the researchers designed a specialized, lightweight network (3DCellSegNet) that achieves comparable or better performance with significantly higher efficiency.
  • Targeted Loss for Clumped Cells: The "clumped cell problem" is a major hurdle in segmentation. The 3DCellSeg Loss directly tackles this by specifically penalizing uncertain predictions near cell boundaries, which is where cells tend to merge. This is a more nuanced approach than standard loss functions.
  • Simplicity is Key for Robustness: The entire 3DCellSeg pipeline requires only one manually tuned hyperparameter (minimum touching area in TASCAN). This drastically simplifies adaptation to new datasets and improves robustness, a significant advantage over methods needing many hyperparameters.
  • Transfer Learning Works: The model demonstrated good performance even when fine-tuned on a very small subset of data from a different domain, highlighting its transferable nature.

Key Results

  • Performance: 3DCellSeg outperformed baseline models on the HMS, ATAS, and LRP datasets, achieving overall accuracies of 76.4%, 95.6%, and 74.7%, respectively. It achieved comparable performance on the Ovules dataset (82.2% accuracy).
  • Efficiency: 3DCellSegNet is significantly smaller and faster than other deep learning backbones like U-Net and VoxResNet, with inference times reduced by 50-70%. TASCAN is also much faster than DBSCAN for instance segmentation, reducing processing time from hours to minutes for large images.
  • Robustness: Ablation studies confirmed that 3DCellSegNet, 3DCellSeg Loss, and TASCAN each contribute to the overall accuracy and robustness of the pipeline. The model showed consistent performance across diverse cell types and datasets.

Limitations and Future Work

  • Limitations: While the paper addresses the clumped cell problem, it acknowledges that membrane voxels can still be misclassified, leading to some segmentation errors. The current approach uses deep learning only for semantic segmentation, not end-to-end instance segmentation.
  • Future Work: The authors propose developing an end-to-end deep learning model that directly outputs 3D cell instances. They also aim to incorporate more domain-specific knowledge about cell characteristics (like size and distribution) to further improve accuracy and classification.

Summary updated

Abstract

Cell segmentation plays a crucial role in understanding, diagnosing, and treating diseases. Despite the recent success of deep learning-based cell segmentation methods, it remains challenging to accurately segment densely packed cells in 3D cell membrane images. Existing approaches also require fine-tuning multiple manually selected hyperparameters on the new datasets. We develop a deep learning-based 3D cell segmentation pipeline, 3DCellSeg, to address these challenges. Compared to the existing methods, our approach carries the following novelties: (1) a robust two-stage pipeline, requiring only one hyperparameter; (2) a light-weight deep convolutional neural network (3DCellSegNet) to efficiently output voxel-wise masks; (3) a custom loss function (3DCellSeg Loss) to tackle the clumped cell problem; and (4) an efficient touching area-based clustering algorithm (TASCAN) to separate 3D cells from the foreground masks. Cell segmentation experiments conducted on four different cell datasets show that 3DCellSeg outperforms the baseline models on the ATAS (plant), HMS (animal), and LRP (plant) datasets with an overall accuracy of 95.6%, 76.4%, and 74.7%, respectively, while achieving an accuracy comparable to the baselines on the Ovules (plant) dataset with an overall accuracy of 82.2%. Ablation studies show that the individual improvements in accuracy is attributable to 3DCellSegNet, 3DCellSeg Loss, and TASCAN, with the 3DCellSeg demonstrating robustness across different datasets and cell shapes. Our results suggest that 3DCellSeg can serve a powerful biomedical and clinical tool, such as histo-pathological image analysis, for cancer diagnosis and grading.
BibTeX
@inproceedings{wang2022novel,
  author = {Andong Wang and Qi Zhang and Yang Han and Sean G. Megason and Sahand Hormoz and Kishore Mosaliganti and Jacqueline C. K. Lam and Victor O. K. Li},
  title = {{A novel deep learning-based 3D cell segmentation framework for future image-based disease detection}},
  booktitle = {Scientific},
  year = {2022},
  doi = {10.1038/s41598-021-04048-3},
}
Figure