Deep learning detects virus presence in cancer histology

Jakob Nikolas Kather, Jefree J. Schulte, Heike I. Grabsch, Chiara Maria Lavinia Loeffler, Hannah Sophie Muti, James M. Dolezal, Andrew Srisuwananukorn, Nishant Agrawal, Sara Kochanny, Saskia von Stillfried, Peter Boor, Takaki Yoshikawa, Dirk Jaeger, Christian Trautwein, Peter Bankhead, Nicole A. Cipriani, Tom Luedde, Alexander T. PearsonbioRxiv 2019 · DOI

TL;DR

This paper presents a deep learning approach that can detect the presence of oncogenic viruses (HPV and EBV) in cancer histology images, using standard H&E stains. This method offers a potentially faster and cheaper alternative to current molecular tests, and importantly, the researchers developed a way to visualize the AI's decision-making process, making it more interpretable for pathologists.

What Problem Does This Paper Address?

Certain cancers are caused by oncogenic viruses like Human Papillomavirus (HPV) and Epstein-Barr Virus (EBV). Identifying these viral origins is crucial because virus-associated tumors often respond differently to treatments like chemotherapy, radiation, and immunotherapy. For example, HPV-positive head and neck cancers tend to have better survival rates and may benefit from less aggressive treatment. However, current methods for detecting these viruses, such as molecular tests (PCR, in-situ hybridization, DNA sequencing), are often expensive, time-consuming, and not universally available. This creates a need for a more accessible and cost-effective diagnostic tool.

Methods

The core of this study utilizes deep transfer learning, specifically a modified VGG19 convolutional neural network. This network was pre-trained on the ImageNet dataset and then fine-tuned for the task of classifying cancer histology images. The researchers processed whole slide images by tiling them into smaller sections (256x256 µm²), which were then downsampled for the network.

A key innovation is the reverse-engineering of the trained neural networks using a Deep Dream algorithm. This technique allows the researchers to generate "pseudo-images" that highlight the morphological features the AI found most important for classifying a tile as virus-positive or virus-negative. This "feature visualization" aims to make the AI's decisions more understandable to human pathologists.

Datasets

The study used data from two major sources:

  • The Cancer Genome Atlas (TCGA):
    • Head and Neck Squamous Cell Carcinoma (HNSC): N=412 patients (12% HPV positive)
    • Gastric Adenocarcinoma (STAD): N=317 patients (8% EBV positive)
  • In-house cohorts:
    • UCH-HNSC (University of Chicago): N=105 patients (49% HPV positive)
    • KCCH-STAD (Kanagawa Cancer Center): N=197 patients (5% EBV positive)

The study also mentions using external validation sets from these in-house cohorts.

Key Insights and Ideas

  • Histology as a Biomarker: The study demonstrates that routine Hematoxylin and Eosin (H&E) stained histology slides, which are readily available, contain sufficient morphological information for deep learning models to detect the presence of oncogenic viruses.
  • Bridging the Gap: The developed deep learning workflow offers a potential low-cost and fast alternative to expensive and time-consuming molecular tests for virus detection.
  • Explainable AI in Pathology: The use of Deep Dream for feature visualization is a significant step towards making AI in pathology more transparent. It allows pathologists to see what the AI is looking at, fostering trust and enabling quality control.
  • Plausible Morphological Features: The visualized features identified by the AI align with known or plausible morphological characteristics associated with virus-driven cancers, suggesting the AI is learning biologically relevant patterns.

Key Results

  • HPV Detection in HNSC: Achieved a patient-level 3-fold cross-validated Area Under the Curve (AUC) of 0.89 [0.82; 0.94] on TCGA data.
  • EBV Detection in Gastric Cancer: Achieved a patient-level 3-fold cross-validated AUC of 0.80 [0.70; 0.92] on TCGA data.
  • External Validation: The EBV classifier achieved an AUC of 0.81 on the KCCH-STAD external validation set. The HPV classifier achieved an AUC of 0.70 on the UCH-HNSC external validation set.
  • Feature Visualization: Pathologists described the Deep Dream generated images as "beautiful" and "psychedelic," and were able to relate them to specific morphological features like "sheets of small nodules" (HPV negative HNSC) or "large nests with rounded borders" (HPV positive HNSC).

Limitations and Future Work

  • Performance vs. Gold Standard: While the deep learning approach shows promising accuracy, its sensitivity and specificity are currently lower than established gold-standard molecular tests (e.g., p16 IHC for HPV, EBER ISH for EBV).
  • Noisy Tile-Level Data: Individual tile-level predictions can be noisy, with false positives sometimes showing lymphocyte-rich stroma, which is a known hallmark of virus-driven cancers, making misclassifications plausible.
  • External Validation Drop: The performance drop in the UCH-HNSC external validation cohort for HPV detection might be due to the smaller cohort size or differences in the gold standard used for virus status determination and the prevalence of HPV in the training vs. testing sets.
  • Future Work:
    • Training on larger clinical cohorts is expected to improve performance.
  • Further optimization of hyperparameters and neural network architectures.
  • Dividing classifiers by anatomical sub-sites of tumors could enhance performance.
  • The image-based biomarker needs to be tested in prospective clinical trials before widespread clinical use.
  • The "human to AI and back" approach, using feature visualization, can be further explored for quality control and to gain new insights into histomorphology.

Summary updated

Abstract

Abstract Oncogenic viruses like human papilloma virus (HPV) or Epstein Barr virus (EBV) are a major cause of human cancer. Viral oncogenesis has a direct impact on treatment decisions because virus-associated tumors can demand a lower intensity of chemotherapy and radiation or can be more susceptible to immune check-point inhibition. However, molecular tests for HPV and EBV are not ubiquitously available. We hypothesized that the histopathological features of virus-driven and non-virus driven cancers are sufficiently different to be detectable by artificial intelligence (AI) through deep learning-based analysis of images from routine hematoxylin and eosin (HE) stained slides. We show that deep transfer learning can predict presence of HPV in head and neck cancer with a patient-level 3-fold cross validated area-under-the-curve (AUC) of 0.89 [0.82; 0.94]. The same workflow was used for Epstein-Barr virus (EBV) driven gastric cancer achieving a cross-validated AUC of 0.80 [0.70; 0.92] and a similar performance in external validation sets. Reverse-engineering our deep neural networks, we show that the key morphological features can be made understandable to humans. This workflow could enable a fast and low-cost method to identify virus-induced cancer in clinical trials or clinical routine. At the same time, our approach for feature visualization allows pathologists to look into the black box of deep learning, enabling them to check the plausibility of computer-based image classification.
BibTeX
@misc{kather2019deep,
  author = {Jakob Nikolas Kather and Jefree J. Schulte and Heike I. Grabsch and Chiara Maria Lavinia Loeffler and Hannah Sophie Muti and James M. Dolezal and Andrew Srisuwananukorn and Nishant Agrawal and Sara Kochanny and Saskia von Stillfried and Peter Boor and Takaki Yoshikawa and Dirk Jaeger and Christian Trautwein and Peter Bankhead and Nicole A. Cipriani and Tom Luedde and Alexander T. Pearson},
  title = {{Deep learning detects virus presence in cancer histology}},
  year = {2019},
  doi = {10.1101/690206},
  howpublished = {bioRxiv},
}
Figure