Abstract
Fluorescent microscopy imaging is vital to capturing single-cell spatial data, characterizing tissue organization and facilitating comprehensive analysis of cellular state. Advancements in fluorescent microscopy imaging technologies have enabled precise downstream cellular analysis, particularly in cell segmentation. Accurate segmentation of individual cells allows better profiling and understanding of cell properties and behaviors. The majority of existing segmentation methods predominantly concentrate on enhancing segmentation algorithms, and their effectiveness strongly relies on the input stained image quality. Factors such as high cellular density, indistinct cell boundaries, and staining artifacts can result in uneven and low-quality staining, particularly causing missing or unclear membrane staining. These artifacts adversely impact the efficacy of the subsequent cell segmentation methods. To tackle this insufficient membrane staining, we propose a novel approach, Mem-GAN, to generate high-quality membranes for cells with missing or weak membranes. Inspired by advanced style transfer techniques in computer vision, Mem-GAN styles the content of the cells with missing or weak membranes into cells with integrated membrane staining. Considering the differences in membrane morphology between epithelial/tumor cells and immune cells, Mem-GAN deals with tumor and immune cells separately, not only enhancing membrane staining for cells with partially weak membrane signals but also generating membranes for cells with only nuclear channels. The proposed Mem-GAN is evaluated using the publicly available CosMx dataset. Experimental results demonstrate significant improvements in image staining quality, more accurate representation of membrane morphology characteristics, and better performance in downstream segmentation tasks. Mem-GAN is flexibly adapted and applied to other spatially resolved transcriptomics datasets, such as MERFISH and FISHseq. Our work provides a new perspective on tackling the challenges in cell segmentation from fluorescent microscopy image restoration.
Introduction
In the field of spatial biology, accurate cell segmentation—the process of identifying the boundaries of individual cells—is the critical foundation for downstream analysis. To achieve this, researchers typically rely on fluorescent stains: DAPI for the nucleus and various antibodies (such as PanCK or CD3) for the cell membrane.
However, real-world imaging is rarely perfect. High cellular density, optical blur, and chemical artifacts often lead to "missing" or "weak" membrane signals. While most existing research attempts to improve segmentation by making algorithms (like U-Nets) more complex, MEM-GAN addresses the root cause: input data quality. By utilizing style transfer to "hallucinate" high-quality pseudo-membranes, MEM-GAN provides a robust pre-processing solution that significantly boosts the accuracy of standard segmentation tools.
Methods
MEM-GAN is built upon a Cycle-consistent Generative Adversarial Network (CycleGAN) architecture. The primary advantage of using CycleGAN is its ability to handle unpaired data. In biological imaging, it is impossible to obtain the exact same cell with both a poor stain and a perfect stain. CycleGAN allows the model to learn the "style" of well-stained membranes from one set of images and apply it to the "content" (the nuclei positions) of poorly stained images.
Key technical components of the framework include:
- Morphology-Aware Training: Recognizing that biological morphology is diverse, separate models are trained for tumor cells (irregular and jagged) and immune cells (rounded and uniform).
- Image Restoration and Imputation: The model can either enhance a weak existing signal or "impute" a membrane from scratch using only the nuclear (DAPI) channel.
- Biological Fidelity: A post-processing step merges the original, genuine nuclear signals with the generated membranes to ensure the underlying biological data remains uncorrupted.
Datasets
The primary evaluation of MEM-GAN was conducted using the CosMx dataset, featuring non-small cell lung cancer (NSCLC) tissue with approximately 390,000 cells. The authors also demonstrated the flexibility of the framework by adapting it to other spatially resolved transcriptomics datasets, including MERFISH and FISHseq.
Results
The framework was evaluated both quantitatively and through expert qualitative assessment. Using the Intersection over Union (IoU) metric, MEM-GAN showed an average improvement of 10% in segmentation accuracy. In specific challenging samples, the IoU increased from 0.28 to 0.35.
Visual inspections demonstrate that MEM-GAN successfully "fills in the gaps" in areas where membrane staining was previously invisible, providing the necessary spatial constraints for tools like Cellpose to identify distinct cell territories. Furthermore, in a human evaluation study, five PhD-level experts rated the generated images' "Membrane Completeness" at 3.42 on a 5-point scale, compared to just 1.97 for the original low-quality images.
Discussion and Conclusion
MEM-GAN shifts the focus from the segmentation "engine" to the "fuel." By improving image quality, it makes existing tools much more effective without requiring the retraining of segmentation models themselves. While the current pipeline requires some manual effort for patch categorization, future work aims to automate this process and explore Diffusion Models for even higher-fidelity membrane generation.
Ultimately, MEM-GAN provides a new perspective on tackling the challenges of cell segmentation through fluorescent microscopy image restoration. The implementation is open-source and available at https://github.com/OmicsML/Mem-GAN.
Summary updated