Classification of benign and malignant subtypes of breast cancer histopathology imaging using hybrid CNN-LSTM based transfer learning

Mahati Munikoti Srikantamurthy, V. P. Subramanyam Rallabandi, Dawood Babu Dudekula, Sathishkumar Natarajan, Junhyung ParkBMC Medical Imaging 2023 · DOI

TL;DR

This paper introduces a novel hybrid Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) model for classifying breast cancer subtypes from histopathology images. By leveraging transfer learning from ImageNet and optimizing with the Adam optimizer, the model achieves state-of-the-art accuracy in distinguishing benign from malignant tumors and their subtypes, paving the way for more efficient and accurate cancer diagnosis.

What Problem Does This Paper Address?

Accurately classifying breast cancer subtypes from histopathology images is crucial for effective diagnosis and treatment. However, this process is currently labor-intensive, requiring expert pathologists to manually examine numerous slides, which can be time-consuming and prone to human error. Existing automated methods often struggle with the complexity and variability of histopathology images, necessitating more robust and accurate deep learning approaches. This paper aims to bridge this gap by developing an automated system that can reliably classify benign and malignant breast cancer subtypes with high accuracy.

Methods

The core of this research is a hybrid CNN-LSTM model. The CNN component, utilizing transfer learning from pre-trained ImageNet weights (specifically InceptionResNetV2 or ResNet50), acts as a powerful feature extractor. These extracted features are then fed into an LSTM recurrent neural network module, which is adept at capturing sequential information and temporal dependencies within the data. The outputs from both modules are merged using element-wise multiplication before being passed through a final classification layer with a SoftMax activation function to predict the cancer subtype.

Key methodological aspects include:

  • Transfer Learning: The use of pre-trained ImageNet weights significantly reduces the need for massive datasets and speeds up the training process by leveraging knowledge gained from a large-scale image recognition task.
  • Hybrid CNN-LSTM Architecture: Combining the spatial feature extraction capabilities of CNNs with the sequential modeling power of LSTMs allows for a more comprehensive analysis of the histopathology images.
  • Data Augmentation: Techniques like random intensity variation, rotation, flipping, and translation were employed to artificially increase the size and diversity of the training dataset, mitigating issues of class imbalance and preventing overfitting.
  • Model Optimization: The study explored various optimizers (Adam, RMSProp, SGD) and hyperparameters, concluding that Adam with specific learning rates and epoch counts provided the best performance without overfitting.

Datasets

The study utilized the BreakHis (Breast Cancer Histopathological Image Classification) dataset. This public dataset comprises 7909 microscopic images of breast tumor tissue from 82 patients, acquired at four different magnification factors: 40×, 100×, 200×, and 400×. The dataset contains 2480 benign samples and 5429 malignant samples, categorized into four benign subtypes (adenosis, fibroadenoma, phyllodes tumor, tubular adenoma) and four malignant subtypes (ductal carcinoma, lobular carcinoma, mucinous carcinoma, papillary carcinoma).

Key Insights and Ideas

  • Synergy of CNN and LSTM: The combination of CNNs for spatial feature extraction and LSTMs for sequential pattern recognition proved highly effective for histopathology image classification, outperforming individual CNN models.
  • Importance of Transfer Learning: Leveraging pre-trained models on ImageNet significantly boosted performance and reduced training time, demonstrating its value in medical imaging tasks where large annotated datasets can be scarce.
  • Adam Optimizer's Efficacy: The Adam optimizer consistently yielded the best results, achieving maximum accuracy and minimum model loss across different magnifications and classification tasks.
  • Data Augmentation's Crucial Role: Effective data augmentation was essential for handling class imbalance and improving the model's generalization capabilities, leading to more robust predictions.

Key Results

The proposed hybrid CNN-LSTM model demonstrated impressive performance:

  • Binary Classification (Benign vs. Malignant): Achieved an overall accuracy of 99%. Specific accuracies for different magnifications were: 40× (99.03%), 100× (99.75%), 200× (99.64%), and 400× (98.07%).
  • Multi-class Classification (Subtypes): Achieved an overall accuracy of 92.5%. Specific accuracies for different magnifications were: 40× (96.3%), 100× (92.6%), 200× (88.04%), and 400× (92.51%).
  • Comparison with State-of-the-Art: The model outperformed existing methods in both binary and multi-class classification tasks, as evidenced by comparisons in Table 5.
  • High AUC Scores: Precision-Recall (PR) curves showed high areas under the curve (AUC) for binary classification (e.g., 0.9990 for 200x) and Receiver Operating Characteristic (ROC) curves also indicated excellent performance (e.g., 0.9953 for 200x).

Limitations and Future Work

The authors acknowledge several limitations:

  • Computational Expense: The model, while effective, is computationally intensive and can be complex to implement on standard workstations.
  • Interpretability: Like many deep learning models, the "black-box" nature of the CNN-LSTM makes it challenging to fully interpret why specific predictions are made, which is crucial for clinical trust.
  • Limited Magnification Range: The current model is optimized for 40×, 100×, 200×, and 400× magnifications. Further research is needed to adapt it for other magnification levels.
  • No Temporal Prediction: The model cannot predict cancer stage progression or stability over time, requiring longitudinal data.

Future work could focus on developing more interpretable models, extending the model to handle a wider range of magnifications, and potentially incorporating temporal analysis for prognosis. The authors also suggest that the proposed method's feasibility extends to classifying other cancer types and diseases.

Summary updated

BibTeX
@article{srikantamurthy2023classification,
  author = {Mahati Munikoti Srikantamurthy and V. P. Subramanyam Rallabandi and Dawood Babu Dudekula and Sathishkumar Natarajan and Junhyung Park},
  title = {{Classification of benign and malignant subtypes of breast cancer histopathology imaging using hybrid CNN-LSTM based transfer learning}},
  journal = {BMC Medical Imaging},
  year = {2023},
  doi = {10.1186/s12880-023-00964-0},
}
Figure