Classification of breast cancer histology images using transfer learning

Sulaiman Vesal, Nishant Ravikumar, AmirAbbas Davari, Stephan Ellmann, Andreas MaierarXiv 2018 · arXiv

TL;DR

This paper presents a robust transfer-learning approach to automate the classification of breast cancer histology images into four critical categories: normal, benign, in situ carcinoma, and invasive carcinoma. By fine-tuning pre-trained deep learning models (ResNet50 and Inception-V3), the researchers achieved a remarkable 97.5% classification accuracy, demonstrating the potential for AI to reduce subjectivity and workload in clinical pathology.

What Problem Does This Paper Address?

Breast cancer diagnosis heavily relies on histopathology—the microscopic examination of tissue samples stained with Hematoxylin and Eosin (H&E). However, this process is notoriously "laborious and highly subjective," often leading to significant variability between different pathologists.

The challenge for computer-aided diagnosis (CAD) systems in this field is two-fold:

  1. Data Scarcity: High-quality, annotated medical datasets are often small compared to general-purpose computer vision datasets.
  2. Visual Inconsistency: Variations in slide preparation, staining protocols, and scanning equipment lead to significant color differences between images, which can confuse standard machine learning models.

Methods

The authors proposed a multi-step workflow to overcome these challenges:

  1. Stain Normalization: To handle color variability, they employed the Reinhard method, which matches the color statistics of a source image to a target image in the LAB color space. This ensures the model learns biological structures rather than lab-specific staining quirks.

    Fig. 1: Examples of histology images from each class before (top row) and after (bottom row) stain normalization.
  2. Patch-Based Processing: Because histology images are extremely high-resolution (2040 × 1536 pixels), the authors extracted 512 × 512 patches with 50% overlap. This allows the model to focus on localized nuclear organization and tissue architecture.

  3. Data Augmentation: To simulate the arbitrary orientation of slides under a microscope, they applied rigid transformations (rotations and flips), expanding their dataset to over 67,000 patches.
  4. Transfer Learning: Instead of training from scratch, they fine-tuned two heavyweights of computer vision: Google’s Inception-V3 and ResNet50, both pre-trained on the massive ImageNet database.
  5. Majority Voting: To classify a whole image, the model predicts labels for all extracted patches, and the final diagnosis is determined by a majority vote.
Fig. 2: Breast histology image classification workflow by fine-tuning Google‘s Inception-V3 and ResNet50 network architectures. The block on the left repre- sents the pre-processing steps and the blocks on the right depict the Inception-V3 (top) and ResNet50 (bottom) network architectures.

Datasets

The study utilized the BACH 2018 (Breast Cancer Histology) grand challenge dataset. It consists of 400 high-resolution H&E stained microscopy images, evenly split (100 each) into four classes: normal tissue, benign lesion, in situ carcinoma, and invasive carcinoma. The data was partitioned into 60% for training, 20% for validation, and 20% for testing.

Key Insights and Ideas

  • Residual Learning Superiority: The ResNet50 architecture, which uses "shortcut connections" to learn residual mappings, significantly outperformed Inception-V3. This suggests that residual learning is particularly effective at capturing the complex, fine-grained textures found in histopathology.
  • ImageNet Knowledge Transfers: A key "aha moment" is that features learned from everyday objects (like cats, dogs, and cars in ImageNet) are highly transferable to microscopic medical images. The low-level edge and texture detectors learned on general data provide a powerful foundation for domain-specific medical fine-tuning.
  • Preprocessing is Non-Negotiable: The emphasis on stain normalization highlights that in medical AI, data cleaning is just as important as the model architecture itself.

Key Results

The quantitative results were decisive:

  • ResNet50 achieved a test classification accuracy of 97.50% for the four-class task.
  • Inception-V3 trailed behind with an accuracy of 91.25%.
  • ROC Analysis: ResNet50 showed near-perfect performance across all classes, with an Area Under the Curve (AUC) of 1.00 for most categories, indicating extremely high sensitivity and specificity.
Table 1: Patch-wise and image-wise classification accuracy(%) for the ResNet50 and Inception-V3 networks.
Fig. 3: ROC curves for unseen test set using Google‘s Inception-V3 and ResNet50 fine-tuned architectures.

Limitations and Future Work

While the results are impressive, the authors acknowledge the limitations of working with a relatively small dataset of 400 images. While patch extraction and augmentation help, larger-scale datasets would be required to ensure the model generalizes across the vast diversity of clinical cases seen worldwide.

Future research directions include exploring even deeper architectures and potentially integrating these CAD systems into real-time pathology workflows to assist specialists in making faster, more objective diagnoses.

Summary updated

Abstract

Breast cancer is one of the leading causes of mortality in women. Early detection and treatment are imperative for improving survival rates, which have steadily increased in recent years as a result of more sophisticated computer-aided-diagnosis (CAD) systems. A critical component of breast cancer diagnosis relies on histopathology, a laborious and highly subjective process. Consequently, CAD systems are essential to reduce inter-rater variability and supplement the analyses conducted by specialists. In this paper, a transfer-learning based approach is proposed, for the task of breast histology image classification into four tissue sub-types, namely, normal, benign, \textit{in situ} carcinoma and invasive carcinoma. The histology images, provided as part of the BACH 2018 grand challenge, were first normalized to correct for color variations resulting from inconsistencies during slide preparation. Subsequently, image patches were extracted and used to fine-tune Google`s Inception-V3 and ResNet50 convolutional neural networks (CNNs), both pre-trained on the ImageNet database, enabling them to learn domain-specific features, necessary to classify the histology images. The ResNet50 network (based on residual learning) achieved a test classification accuracy of 97.50% for four classes, outperforming the Inception-V3 network which achieved an accuracy of 91.25%.
BibTeX
@misc{vesal2018classification,
  author = {Sulaiman Vesal and Nishant Ravikumar and AmirAbbas Davari and Stephan Ellmann and Andreas Maier},
  title = {{Classification of breast cancer histology images using transfer learning}},
  year = {2018},
  eprint = {1802.09424},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1802.09424},
}

Loading related content...

Figure