Cell Segmentation in Multi-modality High-Resolution Microscopy Images with Cellpose

Hyungjo Byun, Kwanyoung Lee, Hyunjung ShimNeurIPS 2022 · OpenReview

TL;DR

Researchers have successfully adapted the Cellpose model to tackle the "Weakly Supervised Cell Segmentation in Multi-modality High-Resolution Microscopy Images" challenge. By standardizing input channels and implementing a robust diameter estimation technique, they achieved a significant performance boost, raising the F1 score from a baseline of 0.3112 to 0.7607.

Introduction: The Generalization Gap in Cell Segmentation

Cell segmentation—identifying the boundaries of individual cells in microscopic images—is a cornerstone of biological research. While deep learning has made huge strides in this domain, most models suffer from a "generalization gap." A model trained on fluorescent images often fails when applied to brightfield or phase-contrast images.

This lack of generalization stems from three main issues:

  1. Modality Diversity: Different staining methods and microscope types (e.g., DIC vs. fluorescent) produce wildly different visual textures and channel counts.
  2. Morphological Complexity: Cells are not always perfectly circular; they can be elongated, packed tightly, or possess irregular boundaries.
  3. Resolution Variance: Microscopy images can range from standard $512 \times 512$ pixels to massive $10,000 \times 10,000$ resolutions, making it difficult for a model to determine the appropriate scale of a cell.

Methodology: Adapting Cellpose

The authors chose Cellpose as their baseline architecture. Unlike standard U-Nets that simply predict "cell vs. background" pixels, Cellpose predicts a gradient vector field where vectors point toward the center of the cell. This "flow" approach is highly effective at separating touching cells and handling irregular shapes.

To adapt Cellpose for this specific challenge, the researchers introduced two key modifications:

  • Adaptive Segmentation Channel: Cellpose typically expects two channels: cytoplasm and nucleus. Because the challenge dataset featured varying channel counts, the authors standardized the input. They mapped green to the first channel (cytoplasm) and blue to the second (nucleus). For single-channel images, the second channel was padded with zeros.
  • Retrained Size Estimation: To handle massive resolution differences, they implemented a size estimation model. This model uses a "style vector" (an encoding of the image's visual style) to predict the average cell diameter. The image is then resized to a standard scale before segmentation, ensuring the model's receptive field matches the cell size.

Dataset

The study utilized the dataset from the "Weakly Supervised Cell Segmentation in Multi-modality High-Resolution Microscopy Images" challenge.

  • Composition: 1,000 image patches in total, including 300 brightfield, 300 fluorescent, 200 phase-contrast, and 200 differential interference contrast (DIC) images.
  • Cell Types: Includes red and white blood cells, plasma cells, Hanseniaspora, and various animal tissue cells.
  • Resolution: Ranges broadly from $512 \times 512$ to $10,000 \times 10,000$ pixels.

Key Insights

  • Flow is Better than Polygons: The authors highlight that methods like Stardist (which uses star-convex polygons) struggle with elongated cells. Cellpose’s gradient flow approach is more flexible and does not assume a specific geometric shape.
  • Standardization over Complexity: Interestingly, attempting to "merge" all available channels into the input actually decreased performance (yielding an F1 of 0.7336). Sticking to a fixed green/blue mapping proved much more robust for generalization.
  • The Resolution Trap: Without diameter estimation, the F1 score dropped by over 12%. This proves that for high-resolution microscopy, analyzing the image at the correct scale is just as critical as the neural network architecture itself.

Results

The quantitative improvements achieved by the proposed modifications were dramatic, as shown in the ablation study below.

Ablation study results showing F1 score improvements

By adding fine-tuning, adaptive channels, and diameter estimation, the model more than doubled the baseline performance, reaching a final F1 score of 0.7607.

Qualitatively, the model demonstrated high resilience to non-circular cell shapes and successfully recognized cells in modalities it had not specifically seen in the original Cellpose pre-training set.

Qualitative comparison between the baseline and proposed model

Limitations and Future Work

  • The "Bacilli" Problem: The model struggled with rod-shaped bacteria (bacilli), which constituted only 0.3% of the training data. The authors noted that the model often flagged these shapes as "uncertain."
  • Hardware Constraints: The Cellpose backbone requires uploading all images to RAM during preprocessing. This created significant bottlenecks and prevented the authors from experimenting with the full unlabeled dataset provided by the challenge.
  • Future Directions: The team plans to explore semi-supervised learning using consistency regularization to better utilize unlabeled data and improve the recognition of rare cell morphologies like bacilli.

Summary updated

Abstract

Deep learning has achieved great results in microscopy image processing in the field of Biology. However, a generalized model is needed which can solve overfitting and produce good performance for test images and unseen classes. The reason why it is difficult to make a generalized model is because of the diversity of modality, staining methods, cell shapes and resolution of microscopy images. The dataset of the "Weakly Supervised Cell Segmentation in Multi-modality High-Resolution Microscopy Images" challenge consists of images with these diverse characteristics. Therefore, we trained cellpose to perform instance segmentation well on dataset having various characteristics. In order to apply the cellpose model to the challenge dataset, we designated model to always use green and blue channels for any type of images. What’s more we newly created the size estimation model predicting diameter of cell to operate on various resolutions. As a result, we could achieve F1 score 0.7607 for the validation (Tuning) set.
BibTeX
@inproceedings{byun2022cell,
  author = {Hyungjo Byun and Kwanyoung Lee and Hyunjung Shim},
  title = {{Cell Segmentation in Multi-modality High-Resolution Microscopy Images with Cellpose}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=-N9jH73x8n5},
}
Figure