Abstract
Analyzing the dynamic changes of cellular morphology is important for understanding the various functions and characteristics of live cells, including stem cells and metastatic cancer cells. To this end, we need to track all points on the highly deformable cellular contour in every frame of live cell video. Local shapes and textures on the contour are not evident, and their motions are complex, often with expansion and contraction of local contour features. The prior arts for optical flow or deep point set tracking are unsuited due to the fluidity of cells, and previous deep contour tracking does not consider point correspondence. We propose the first deep learning-based tracking of cellular (or more generally viscoelastic materials) contours with point correspondence by fusing dense representation between two contours with cross attention. Since it is impractical to manually label dense tracking points on the contour, unsupervised learning comprised of the mechanical and cyclical consistency losses is proposed to train our contour tracker. The mechanical loss forcing the points to move perpendicular to the contour effectively helps out. For quantitative evaluation, we labeled sparse tracking points along the contour of live cells from two live cell datasets taken with phase contrast and confocal fluorescence microscopes. Our contour tracker quantitatively outperforms compared methods and produces qualitatively more favorable results.
Introduction: The Challenge of Cellular Morphodynamics
Understanding cellular morphodynamics—the way a cell changes its shape—is critical for studying cancer metastasis, immune responses, and stem cell development. To quantify these processes, researchers require "dense correspondences," meaning they must track exactly where a specific point on the cell's boundary in one frame moves in the subsequent frame.
This task is exceptionally difficult due to three primary factors:
- Visual Ambiguity: Cell membranes often lack distinct features and look identical to neighboring pixels, causing standard optical flow algorithms to fail.
- Fluidity: Cells are viscoelastic materials that stretch, shrink, and merge, requiring a tracking system that can handle a changing number of points.
- Labeling Constraints: Manually labeling thousands of points across hundreds of video frames to create a supervised training dataset is practically impossible.
Proposed Methodology
The researchers introduced a novel tracker that learns from the physical logic of cell movement rather than human-provided labels. The architecture is designed to handle the inherent fluidity of biological samples.
Architecture and Feature Extraction
The model employs a shared encoder (VGG16 combined with a Feature Pyramid Network) to extract visual features from two consecutive frames, $I_t$ and $I_{t+1}$. Instead of processing the entire image, the model uses bilinear sampling to focus specifically on features at the locations of the cell contour.
To bridge the gap between frames, the model utilizes forward and backward cross-attention. This mechanism allows the network to "attend" to both global and local shape changes simultaneously, effectively handling the complex deformations characteristic of live cells.
Unsupervised Learning via Physical Constraints
In the absence of ground-truth labels, the network is trained using two key unsupervised loss functions:
- Mechanical-Normal Loss: Inspired by the physics of membrane protrusion and retraction, this loss forces predicted point movement to be perpendicular (normal) to the cell's contour.
- Cycle Consistency Loss: This serves as a "round-trip" validation. If a point is tracked from Frame $t$ to Frame $t+1$ and then back to Frame $t$, it should return to its original starting position.
Interestingly, the authors found that traditional "linear spring losses" (meant to keep points equidistant) actually hindered performance during rapid expansion, as they prevented the model from capturing natural stretching.
Experimental Results and Insights
The model was evaluated across diverse datasets, including phase-contrast microscopy (9 videos) and confocal fluorescence microscopy (40 videos). To demonstrate generalizability to other viscoelastic materials, the tracker was also tested on swimming jellyfish videos.
Quantitative Performance
The proposed method outperformed traditional mechanical models and existing deep learning trackers like PoST and UFlow. On phase-contrast datasets, the model achieved a Spatial Accuracy (SA) of 0.729 (at a 0.02 threshold), compared to 0.683 for traditional mechanical models.
Key Insights
- Physics-Driven Learning: The Mechanical-Normal loss was identified as the primary driver of accuracy, suggesting that physical constraints are more effective than visual features alone for this task.
- Morphodynamic Mapping: The model successfully generated heatmaps showing clear patterns of protrusion and retraction, providing a level of detail previously obscured by noise.
Conclusion and Future Work
While the tracker represents a significant advancement, it remains dependent on the quality of initial binary segmentation masks. Future work will focus on improving robustness to noisy segmentations and refining the handling of new points that emerge during extreme cellular expansion. The integration of these components into a real-time, end-to-end pipeline remains a goal for future development.
Summary updated