CellGAN: Conditional Cervical Cell Synthesis for Augmenting Cytopathological Image Classification

Zhenrong Shen, Maosong Cao, Sheng Wang, Lichi Zhang, Qian Wang

TL;DR

CellGAN is a specialized generative adversarial network designed to synthesize high-fidelity cytopathological images of cervical cells. By generating realistic training data for rare abnormal cell types, it boosts the accuracy of automated cancer screening classifiers by up to 5.25%, helping overcome the chronic shortage of expert-annotated medical data.

What Problem Does This Paper Address?

Cervical cancer screening relies heavily on the Thin-prep Cytologic Test (TCT). However, analyzing these slides is a "needle in a haystack" problem: Whole Slide Images (WSIs) are massive, and pathologists must find a few suspicious cells among thousands of normal ones.

While deep learning can automate this, these models are data-hungry. Collecting and annotating thousands of examples for every stage of cervical abnormality is incredibly difficult, requiring rare expertise and significant time. Traditional data augmentation (like flipping or rotating images) doesn't add enough diversity to help models learn the nuanced morphological changes—like nucleus-to-cytoplasm ratios—that signal cancer. The authors address this "data bottleneck" by creating a way to synthesize realistic, class-specific cell images from scratch.

Methods

The authors introduce CellGAN, built upon the FastGAN backbone for training stability and efficiency. The architecture features three key innovations:

  1. Non-linear Class Mapping Network: Instead of simply feeding a class label into the generator, CellGAN uses a dedicated network to embed labels into a high-dimensional space. These embeddings then modulate the image features at multiple scales using Adaptive Instance Normalization (AdaIN), allowing the model to precisely control cellular attributes specific to each cell type.
  2. Skip-layer Global Context (SGC) Module: Cervical cells in a sample can be isolated, squeezed, or overlapped. The SGC module captures these long-range spatial dependencies, ensuring the relationship between cells in the synthesized image looks biologically plausible.
  3. Multi-faceted Discriminator: The discriminator uses a PatchGAN architecture to ensure local color and texture fidelity. It also incorporates a projection-based method to match conditional data distributions and uses self-supervised reconstruction decoders as a regularization technique to prevent the model from "collapsing" during training.

Datasets

The study utilized a dataset of 14,477 images (256×256 pixels) collected from three clinical centers. The images were categorized into five squamous cell types based on the Bethesda System (TBS):

  • NILM: Normal/Negative (7,662 images)
  • ASC-US: Atypical squamous cells of undetermined significance (2,275 images)
  • LSIL: Low-grade lesion (2,480 images)
  • ASC-H: Atypical cells, cannot exclude HSIL (1,638 images)
  • HSIL: High-grade lesion (422 images)

Note the significant class imbalance: there are nearly 20 times more normal images than high-grade lesion images, highlighting why synthesis is so valuable for the rarer classes.

Key Insights and Ideas

  • Beyond Diffusion: Interestingly, the paper finds that CellGAN outperforms Latent Diffusion Models (LDM) in this specific domain. While LDMs are currently the "gold standard" for general image generation, they struggled here to create clear cell boundaries and maintain class separability with limited medical data.
  • The Importance of Spatial Context: The SGC module was the "aha moment" for handling cell clusters. Without it, synthesized images often lacked the complex overlapping patterns seen in real clinical samples.
  • Color Fidelity via PatchGAN: One common failure in medical GANs is "color drift." By penalizing image content at the patch level, the authors ensured the synthesized images maintained the characteristic purple/blue staining of real TCT slides.

Key Results

  • Visual Quality: CellGAN achieved a mean Fréchet Inception Distance (FID) of 41.51, significantly lower (better) than BigGAN (47.32) and LDM (60.65).
  • Classification Boost: When synthetic CellGAN images were used to augment the training set:
  • ResNet-34 accuracy improved from 74.30% to 79.55% (+5.25%).
  • DenseNet-121 accuracy improved from 72.10% to 76.15% (+4.05%).
  • Rare Class Performance: The model was particularly effective at capturing the morphological features of HSIL cells, which are the hardest to collect in large quantities.

Limitations and Future Work

Despite its success, the authors note two primary limitations:

  1. Granular Control: While the model can generate a specific type of cell, it cannot yet explicitly control fine-grained attributes like a specific nucleus size or a precise nucleus-cytoplasm ratio on command.
  2. Resolution: The current output is limited to 256×256 pixels. Future work aims to expand this to larger, more complex scenes containing even more cells.
  3. Clinical Integration: The authors suggest that future versions could be used as interactive training tools for junior pathologists to practice identifying rare abnormalities.

Summary updated

Abstract

Automatic examination of thin-prep cytologic test (TCT) slides can assist pathologists in finding cervical abnormality for accurate and efficient cancer screening. Current solutions mostly need to localize suspicious cells and classify abnormality based on local patches, concerning the fact that whole slide images of TCT are extremely large. It thus requires many annotations of normal and abnormal cervical cells, to supervise the training of the patch-level classifier for promising performance. In this paper, we propose CellGAN to synthesize cytopathological images of various cervical cell types for augmenting patch-level cell classification. Built upon a lightweight backbone, CellGAN is equipped with a non-linear class mapping network to effectively incorporate cell type information into image generation. We also propose the Skip-layer Global Context module to model the complex spatial relationship of the cells, and attain high fidelity of the synthesized images through adversarial learning. Our experiments demonstrate that CellGAN can produce visually plausible TCT cytopathological images for different cell types. We also validate the effectiveness of using CellGAN to greatly augment patch-level cell classification performance. Our code and model checkpoint are available at https://github.com/ZhenrongShen/CellGAN.
BibTeX
@inproceedings{shen2023cellgan,
  author = {Zhenrong Shen and Maosong Cao and Sheng Wang and Lichi Zhang and Qian Wang},
  title = {{CellGAN: Conditional Cervical Cell Synthesis for Augmenting Cytopathological Image Classification}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2023},
  url = {https://openreview.net/forum?id=U7RTNDx0Jg},
}

Loading related content...

Figure