TL;DR
This paper introduces a novel deep learning framework for label-free, 3D segmentation of live cells and their organelles using quantitative phase imaging (QPI). This approach allows for long-term monitoring of cellular dynamics without the need for fluorescent labels, opening new avenues for studying cell biology in its native state.
What Problem Does This Paper Address?
Studying the intricate dynamics of live cells and their internal components (organelles) is fundamental to understanding cell biology. However, current imaging techniques often rely on exogenous labeling agents (like fluorescent dyes or proteins). These labels can perturb the cells, limiting the duration of observation and potentially altering cellular behavior, especially for long-term studies. Furthermore, accurately segmenting individual cells and their organelles in 3D from label-free imaging data, which often has low contrast and overlapping refractive indices between different cellular components, remains a significant challenge. This paper aims to overcome these limitations by developing a robust, automated method for 3D segmentation of live cells and their organelles using label-free QPI.
Methods
The core of this work is a deep learning framework built upon a 3D U-Net architecture. This framework comprises two main components:
- Multi-organelle Segmentation Model: This model takes a 3D refractive index (RI) tomogram as input and predicts segmentation masks for four key subcellular organelles: the nucleus, nucleolus, plasma membrane, and lipid droplet. It employs a multi-task learning approach, where it learns to segment all organelles simultaneously. This strategy is beneficial for improving overall performance and significantly reducing training time compared to training separate models for each organelle.
- Cell-by-Cell Segmentation Model: This model leverages the output from the multi-organelle model (specifically, the nucleus and membrane masks) to distinguish individual cells from each other. It uses the nucleus mask as a "seed" to initiate cell separation and the membrane mask to delineate cell boundaries from the background.
The models were trained and evaluated using 3D QPI data. For training, they utilized a patching strategy to manage the high resolution of the 3D RI tomograms, resizing them and then sampling smaller patches for processing. During inference, they reconstructed the full-volume segmentation by stitching these predicted patches.
Datasets
The study utilized 3D QPI images of live cells for training and evaluation.
- Training and Initial Evaluation: 129 3D QPI images of live NIH3T3 cells were used. Ground-truth masks were generated by manual annotation by three expert biologists using ITK-Snap. This dataset was split into 105 training and 24 evaluation sets.
- Generalizability Testing: Four additional cell lines (A549, MDA-MB-231, HeLa, and RAW 264.7), not used for training, were used to confirm the model's generalizability. Nine 3D QPI images were captured for each of these cell lines.
Key Insights and Ideas
- Label-free is Powerful: The ability to perform detailed 3D segmentation of live cells and organelles without any labels is a significant step forward, enabling truly native observation of cellular processes.
- Multi-task Learning for Organelles: Simultaneously learning to segment multiple organelles within a cell proved more effective and efficient than training individual models, suggesting that the model learns valuable inter-organelle relationships.
- Nucleus as a Seed for Cell Segmentation: Using the nucleus as a reliable anchor point for separating individual cells is an intuitive and effective strategy, especially given that most cells possess at least one nucleus.
- RI as a Rich Source of Information: The refractive index distribution, a physical property derived from QPI, provides sufficient information to distinguish various cellular components, even those with similar visual appearances.
- Dynamic Changes in Activated Macrophages: The framework successfully captured and quantified dynamic morphological and biochemical changes in activated RAW 264.7 cells, highlighting the accumulation of lipid droplets and changes in cell volume and surface area.
Key Results
- Quantitative Segmentation Performance: The average Dice score for cell instance segmentation was 0.758, and for membrane segmentation, it was 0.831. The model demonstrated good generalizability across various cell lines, including those not used in training.
- Qualitative Segmentation: The framework produced visually accurate segmentation masks for cell instances and subcellular organelles, preserving morphological features.
- Time-Lapse Analysis of Macrophages: The study successfully applied the framework to track morphological and biochemical changes in LPS-activated RAW 264.7 cells over 8.5 hours. This revealed significant increases in cell volume and surface area, along with changes in refractive index and protein concentration within various organelles, providing insights into the inflammatory response.
Limitations and Future Work
- Limitations:
- The Dice score for lipid droplet segmentation was lower than for other organelles, likely due to their smaller volume, making them more susceptible to small errors in prediction.
- The cell-by-cell segmentation performance was slightly lower than membrane segmentation in some cell lines, particularly those with confluent growth, indicating challenges in distinguishing densely packed cells.
- Future Work:
- Improving Model Performance: The authors suggest exploring Bayesian neural networks and stable learning methodologies for more robust models.
- User Interaction: Incorporating user interaction to refine predicted segmentation masks could further enhance accuracy.
- Cell Lineage Tracking: Combining this framework with cell lineage tracking technology could enable automated monitoring of cell differentiation processes.
- Real-time Analysis: Improving the efficiency of the networks to process large 3D imagery could support real-time analysis.
Summary updated