A Cytology Dataset for Early Detection of Oral Squamous Cell Carcinoma

Garima Jain, Sanghamitra Pati, Mona Duggal, Amit Sethi, Abhijeet Patil, Gururaj Malekar, Nilesh Kowe, Jitender Kumar, Jatin Kashyap, D. K. Rout, Deepali Deepali, Hitesh Hitesh, Nishi Halduniya, S. Selva Kumar, Heena Tabassum, Rupinder Singh Dhaliwal, Sucheta Devi Khuraijam, Sushma Khuraijam, Sharmila Laishram, Simmi Kharb, Sunita Singh, K. Swaminadtan, Ranjana Solanki, Deepika Hemranjani, Shashank Nath Singh, Uma Handa, Manveen Kaur, Surinder K. Singhal, Shivani Kalhan, Rakesh Kumar Gupta, Sathya N. Ravi, D. Pavithra, Sunil Kumar Mahto, Arvind Kumar, Deepali Tirkey, Saurav Banerjee, L. Sreelakshmi

TL;DR

Researchers have released the first large-scale, multi-center oral cytology dataset specifically designed for the early detection of Oral Squamous Cell Carcinoma (OSCC). By providing nearly 40,000 expert-annotated nuclei from 368 whole-slide images across ten different medical centers, this open-access resource aims to bridge the gap between low-cost screening and high-accuracy AI diagnostics in resource-constrained settings.

What Problem Does This Paper Address?

Oral Squamous Cell Carcinoma (OSCC) is a global health crisis, particularly in regions like Asia and Africa. While Stage I OSCC has a survival rate of up to 90%, nearly half of all cases are diagnosed in advanced stages where survival plummets to 5–20%.

The "gold standard" for diagnosis is histopathology (biopsy), but it is invasive, expensive, and requires specialized laboratory infrastructure. Oral brush cytology—a simpler, "swab-like" biopsy—is a promising alternative for mass screening. However, it suffers from two major bottlenecks: a severe shortage of trained cytopathologists to read the slides and high inter-observer variability.

While AI has successfully automated cervical cancer screening (Pap smears), the field of oral cytology has lacked the large, diverse, and annotated datasets necessary to train robust deep-learning models. This paper introduces that missing foundation.

Methods

The researchers established a standardized pipeline for data collection and annotation across ten tertiary medical centers in India:

  1. Sample Collection: Brush biopsies were taken from 234 patients, including both healthy controls and those with visible lesions.
  2. Dual Staining: To ensure model generalization, slides were prepared using two common protocols: Papanicolaou (PAP) and May-Grünwald-Giemsa (MGG).
  3. Digitization: Slides were scanned at 40x magnification using high-resolution scanners, resulting in Whole Slide Images (WSIs) with resolutions as fine as 0.125 µm/pixel.
  4. Expert Annotation: Pathologists identified viable regions and performed meticulous nucleus-level segmentation and classification using QuPath. Each nucleus was assigned a grade (I through IV) based on established cytological criteria.
  5. Baseline Benchmarking: The team trained four state-of-the-art segmentation models—UNet, UNet++, HoverNet, and StarDist—to establish a performance baseline for the community.

Datasets

The study introduces a new benchmark dataset, which is publicly available on Kaggle and Google Drive.

  • Scale: 368 Whole Slide Images (WSIs) totaling ~700GB.
  • Diversity: Data from 10 different hospitals to account for "domain shift" (variations in equipment and local lab practices).
  • Annotations: 858 high-resolution patches containing 39,246 annotated nuclei.
  • Metadata: Includes clinical details like smoking history and family history.
  • Comparison: Unlike existing cervical datasets (like CRIC or Herlev), this is specifically tailored for the unique morphology of oral cells.

Key Insights and Ideas

  • The Multi-Center Advantage: By collecting data from ten different institutions, the dataset captures the "noise" of real-world clinical practice. AI models trained on this are more likely to work in a new hospital than those trained on a single-source dataset.
  • Stain Agnosticism: Including both PAP and MGG stains is critical. These stains look very different under a microscope; a model trained only on one often fails on the other. This dataset forces models to learn features that are robust to staining variations.
  • The "Expert Bottleneck" Solution: The authors highlight that the primary goal isn't necessarily to replace doctors, but to provide a "rough screening" tool. In low-resource areas, AI can flag suspicious slides, allowing the few available pathologists to focus their time on the most critical cases.

Key Results

The researchers evaluated several deep learning architectures for nucleus segmentation and classification:

  • Top Performer: HoverNet emerged as the strongest model, achieving an Aggregated Jaccard Index (AJI) of 0.752 and a Panoptic Quality (PQ) of 0.542.
  • Segmentation vs. Classification: While the models were quite good at finding nuclei (segmentation), they struggled more with classifying them into the higher-risk categories (III and IV).
  • The Imbalance Challenge: Performance for Category I (normal) was significantly higher than for Category IV (malignant). For example, HoverNet achieved an $F_d$ score of 0.835 overall, but class-specific scores for the most advanced stages were notably lower due to the relative rarity of those cells in the dataset.

Limitations and Future Work

The authors acknowledge a few key hurdles:

  • Class Imbalance: There are far more "normal" (Category I) nuclei than "malignant" (Category IV) ones. Future researchers will need to employ data augmentation or specialized loss functions to handle this skew.
  • Manual Labor: The annotation process is incredibly time-intensive (approx. 15 minutes per small patch). The authors suggest that future work could involve semi-automated annotation to scale the dataset even further.
  • Clinical Integration: While the segmentation results are promising, the next step is "slide-level" classification—moving from identifying a single bad cell to giving a definitive "cancer/no-cancer" diagnosis for an entire patient.

Summary updated

Abstract

Oral squamous cell carcinoma OSCC is a major global health burden, particularly in several regions across Asia, Africa, and South America, where it accounts for a significant proportion of cancer cases. Early detection dramatically improves outcomes, with stage I cancers achieving up to 90 percent survival. However, traditional diagnosis based on histopathology has limited accessibility in low-resource settings because it is invasive, resource-intensive, and reliant on expert pathologists. On the other hand, oral cytology of brush biopsy offers a minimally invasive and lower cost alternative, provided that the remaining challenges, inter observer variability and unavailability of expert pathologists can be addressed using artificial intelligence. Development and validation of robust AI solutions requires access to large, labeled, and multi-source datasets to train high capacity models that generalize across domain shifts. We introduce the first large and multicenter oral cytology dataset, comprising annotated slides stained with Papanicolaou(PAP) and May-Grunwald-Giemsa(MGG) protocols, collected from ten tertiary medical centers in India. The dataset is labeled and annotated by expert pathologists for cellular anomaly classification and detection, is designed to advance AI driven diagnostic methods. By filling the gap in publicly available oral cytology datasets, this resource aims to enhance automated detection, reduce diagnostic errors, and improve early OSCC diagnosis in resource-constrained settings, ultimately contributing to reduced mortality and better patient outcomes worldwide.
BibTeX
@misc{jain2025cytology,
  author = {Garima Jain and Sanghamitra Pati and Mona Duggal and Amit Sethi and Abhijeet Patil and Gururaj Malekar and Nilesh Kowe and Jitender Kumar and Jatin Kashyap and D. K. Rout and Deepali Deepali and Hitesh Hitesh and Nishi Halduniya and S. Selva Kumar and Heena Tabassum and Rupinder Singh Dhaliwal and Sucheta Devi Khuraijam and Sushma Khuraijam and Sharmila Laishram and Simmi Kharb and Sunita Singh and K. Swaminadtan and Ranjana Solanki and Deepika Hemranjani and Shashank Nath Singh and Uma Handa and Manveen Kaur and Surinder K. Singhal and Shivani Kalhan and Rakesh Kumar Gupta and Sathya N. Ravi and D. Pavithra and Sunil Kumar Mahto and Arvind Kumar and Deepali Tirkey and Saurav Banerjee and L. Sreelakshmi},
  title = {{A Cytology Dataset for Early Detection of Oral Squamous Cell Carcinoma}},
  year = {2025},
  eprint = {2506.09661},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.09661},
}
Figure