Multicell migration tracking within angiogenic networks by deep learning-based segmentation and augmented Bayesian filtering

Mengmeng Wang, Lee‐Ling Sharon Ong, Justin Dauwels, H. Harry AsadaJournal of Medical Imaging 2018 · DOI

TL;DR

This paper presents a novel automated system for tracking individual cell migration within 3D angiogenic networks using phase-contrast microscopy. By combining deep learning for cell detection with an augmented Bayesian filtering approach that incorporates biological constraints, the system achieves a high association accuracy of 86.4%, outperforming existing methods for this challenging biological problem.

What Problem Does This Paper Address?

Studying cell migration is crucial for understanding various biological processes, including development, wound healing, and cancer. Angiogenesis, the formation of new blood vessels, is a particularly important area where cell migration plays a central role. While 3D microfluidic devices (MFDs) offer a powerful platform for in vitro angiogenesis studies, analyzing the complex cell movements within these environments presents significant challenges. Traditional 2D cell tracking methods are insufficient for 3D structures, and the low temporal resolution (daily imaging) of phase-contrast microscopy in MFDs makes it difficult to accurately track cells that can appear, disappear, or move in and out of focus. Existing automated tracking systems often struggle with these complexities, leading to a need for more robust and accurate methods.

Methods

The proposed system tackles this challenge through a multi-stage approach:

  • Image Registration: To account for slight misalignments between daily images, the system first spatially registers all phase-contrast image sequences. This is achieved by identifying salient features (intersections of posts within the MFD) and using Hough transforms and iterative closest point algorithms to align the images.
  • Vessel Segmentation: To focus on the relevant areas, the system segments the angiogenic vessels. This involves fitting a parabola to the cell-gel interface and then applying morphological operations (imclose, imfill, bwareaopen) to obtain a binary representation of the vessel shape.
  • Cell Detection: This is a key innovation. The authors employ a Convolutional Neural Network (CNN) classifier trained on a large dataset of cell and non-cell templates derived from both phase-contrast and end-point confocal images. This CNN is used to detect cell candidates within the phase-contrast images. The architecture includes convolutional, ReLU, max-pooling, fully connected, dropout, and classification layers, optimized through cross-validation.
  • Augmented Bayesian Filtering (Multiple Hypothesis Kalman Filtering - MHT): To link detected cell candidates across time, the system uses a backward Kalman filter initialized with accurate cell locations from end-point confocal images. This backward approach is crucial given the daily imaging interval. The core novelty lies in augmenting this with Multiple Hypothesis Tracking (MHT) and incorporating biological knowledge:
  • Cell Overtaking Constraint: Cells within angiogenic networks cannot overtake each other, a constraint used to prune unlikely tracks.
  • Cell Proliferation/Migration Distinction: The system accounts for cell increases due to both proliferation and migration from the cell-gel interface, using an empirical equation to estimate the probability of the latter.
  • Focus/Out-of-Focus Handling: The system implicitly handles cells that go out of focus by relying on the end-point confocal images for initialization and by considering the probability of cells appearing or disappearing.

Datasets

The paper utilizes experimental phase-contrast microscopy image sequences of 3D angiogenic networks cultured in microfluidic devices. End-point confocal microscopy images, where cell nuclei are stained, are used for initializing the tracking and for validating cell detection results. The authors also created a dataset of cell and non-cell templates (3228 cell templates and 3372 non-cell templates) derived from these images to train the CNN classifier.

Key Insights and Ideas

  • Leveraging End-Point Confocal Data: Initializing tracking with end-point confocal images is a smart workaround for the challenges of low temporal resolution and cell appearance/disappearance in phase-contrast microscopy. It provides a strong starting point for backward tracking.
  • Biological Constraints are Crucial: Simply applying standard tracking algorithms is insufficient. Incorporating biological knowledge (e.g., no cell overtaking, distinct sources of cell increase) significantly improves tracking accuracy and robustness.
  • CNNs for Robust Cell Detection: Deep learning, specifically CNNs, proves highly effective for detecting cells in challenging phase-contrast images, outperforming traditional methods like PCA and SVM. This is attributed to the CNN's ability to learn more suitable features for cell representation.
  • MHT for Ambiguity Resolution: Multiple Hypothesis Tracking is essential for handling the inherent ambiguities in associating cell candidates over long time intervals, allowing for delayed decision-making and exploration of multiple possibilities.

Key Results

  • Cell Detection Accuracy: The CNN-based cell detection algorithm achieved a precision of 87.4% and a recall of 91.2% for detecting cell candidates in the "case 2: one slice" evaluation, which better reflects the 2D nature of phase-contrast images.
  • Cell Association Accuracy: The proposed multicell tracking system, combining MHT with backward Kalman filtering and biological constraints, achieved an overall association accuracy of 86.4%. This significantly outperforms other tested association methods: nearest-neighborhood association (26.7%), Kalman filter with validation gating (60.1%), and the Hungarian method (53.0%).
  • Qualitative Results: The system successfully generates cell lineage plots that visualize cell proliferation, migration into the gel, and cells going out of focus, providing valuable biological insights.

Limitations and Future Work

  • Out-of-Focus Cells: The paper acknowledges that cells that are consistently out of focus in the phase-contrast images are not detected, even though they might be visible in confocal images. This limits the completeness of tracking for all cells in the 3D volume.
  • Future Work: The authors suggest exploring deep residual learning to further improve cell detection accuracy. They also propose combining cell detection likelihood with Mahalanobis distance for enhanced association and developing mathematical models to predict cell migration based on the obtained lineage information.

Summary updated

Abstract

Cell migration is a key feature for living organisms. Image analysis tools are useful in studying cell migration in three-dimensional (3-D) <i>in vitro</i> environments. We consider angiogenic vessels formed in 3-D microfluidic devices (MFDs) and develop an image analysis system to extract cell behaviors from experimental phase-contrast microscopy image sequences. The proposed system initializes tracks with the end-point confocal nuclei coordinates. We apply convolutional neural networks to detect cell candidates and combine backward Kalman filtering with multiple hypothesis tracking to link the cell candidates at each time step. These hypotheses incorporate prior knowledge on vessel formation and cell proliferation rates. The association accuracy reaches 86.4% for the proposed algorithm, indicating that the proposed system is able to associate cells more accurately than existing approaches. Cell culture experiments in 3-D MFDs have shown considerable promise for improving biology research. The proposed system is expected to be a useful quantitative tool for potential microscopy problems of MFDs.
BibTeX
@article{wang2018multicell,
  author = {Mengmeng Wang and Lee‐Ling Sharon Ong and Justin Dauwels and H. Harry Asada},
  title = {{Multicell migration tracking within angiogenic networks by deep learning-based segmentation and augmented Bayesian filtering}},
  journal = {Journal of Medical Imaging},
  year = {2018},
  doi = {10.1117/1.jmi.5.2.024005},
}

Loading related content...

Figure