Discriminative Subtyping of Lung Cancers from Histopathology Images via Contextual Deep Learning

Benjamin J. Lengerich, Maruan Al-Shedivat, Amir Alavi, Jennifer Williams, Sami Labbaki, Eric P. XingmedRxiv 2020 · DOI

Abstract

Summarizing multiple data modalities into a parsimonious cancer “subtype” is difficult because the most informative representation of each patient’s disease is not observed. We propose to model these latent summaries as discriminative subtypes: sample representations which induce accurate and interpretable sample-specific models for downstream predictions. In this way, discriminative subtypes, which are shared between data modalities, can be estimated from one data modality and optimized according to the predictions induced in another modality. We apply this approach to lung cancer by training a deep neural network to predict discriminative subtypes from histopathology images, and use these predicted subtypes to generate models which classify adenocarcinoma, squamous cell carcinoma, and healthy tissue based on transcriptomic signatures. In this way, we optimize the latent discriminative subtypes through induced prediction loss, and the discriminative subtypes are interpreted with standard interpretation of transcriptomic predictive models. Our framework achieves state-of-the-art classification accuracy (F1-score of 0.97) and identifies discriminative subtypes which link histopathology images to transcriptomic explanations without requiring pre-specification of morphological patterns or transcriptomic processes.

Introduction

Cancer is a highly heterogeneous disease where patients with identical diagnoses often exhibit divergent genetic profiles and clinical trajectories. Effective precision medicine requires clinicians to synthesize high-dimensional data from multiple modalities, such as visual histopathology slides (phenotypic data) and transcriptomic profiles (genotypic data).

Traditional machine learning approaches typically address multimodal data through late fusion (analyzing modalities in isolation) or early fusion (concatenating features into a black-box model). Both methods have significant drawbacks: late fusion ignores cross-modal interactions, while early fusion often sacrifices interpretability. This research introduces "discriminative subtyping," a framework that integrates these complex data types while maintaining human-readable explanations by using one modality to contextualize the interpretation of another.

Methods: Contextual Explanation Networks

The core of this approach is the Contextual Explanation Network (CEN). Unlike standard neural networks that map inputs directly to outputs, a CEN uses "contextual" data to define the parameters of a simpler, interpretable model that processes "predictor" data.

The CEN architecture uses histopathology images as context to select archetypal linear models for transcriptomic data.

The pipeline operates in four stages:

  1. Contextual Encoding: A deep neural network (InceptionV3) analyzes histopathology image patches to output a "discriminative subtype"—a vector of weights.
  2. Dictionary of Archetypes: The system maintains a learned dictionary of linear models (archetypes), each representing a distinct biological "story" or transcriptomic signature.
  3. Sample-Specific Modeling: The discriminative subtype weights are used to softly combine these archetypes into a single, custom linear model tailored to the specific tissue sample.
  4. Transcriptomic Prediction: This custom model processes the patient’s RNA-seq data to perform the final classification into Healthy, Lung Adenocarcinoma (LUAD), or Lung Squamous Cell Carcinoma (LUSC).

Datasets

The study utilized data from the NCI Genomic Data Commons, integrating the TCGA (The Cancer Genome Atlas) and TCIA (The Cancer Imaging Archive). The cohort included 992 patients with paired eosin-stained histopathology slides and RNA-seq profiling. To handle the high resolution of the imaging data, slides were partitioned into over 1.4 million image patches for training, validation, and testing.

Results and Discussion

The CEN framework achieved state-of-the-art performance, significantly outperforming both unimodal baselines and standard multimodal integration techniques.

Performance comparison across patch and sample levels shows CEN outperforming all baselines.

Key Performance Metrics

As shown in the results table, the CEN model reached a macro F1 score of 0.97 and an accuracy of 96.18% at the patch level. Notably, it outperformed "Concatenated" and "Ensemble" models, suggesting that using images to contextualize gene expression is more effective than simple feature merging.

Biological and Morphological Insights

Beyond accuracy, the model provides biological validation. The learned archetypes were found to be largely orthogonal, focusing on distinct pathways. For instance, Archetype 13 was linked to the Smad4 tumor suppressor, while Archetype 26 correlated with apoptosis-related genes like BIRC3 and CASP3. By visualizing the image patches that triggered specific archetypes, the researchers confirmed that the model identifies consistent morphological patterns corresponding to these genetic signatures.

Limitations and Future Work

While promising, the current framework has limitations. It does not yet incorporate clinical variables such as age or smoking history. Additionally, the training process assumes every patch from a cancerous slide is diseased, which introduces label noise. Future work will focus on expanding this framework to predict complex clinical outcomes, including survival rates and treatment responses, across a broader range of cancer types.

Summary updated

Abstract

Abstract Summarizing multiple data modalities into a parsimonious cancer “subtype” is difficult because the most informative representation of each patient’s disease is not observed. We propose to model these latent summaries as discriminative subtypes : sample representations which induce accurate and interpretable sample-specific models for downstream predictions. In this way, discriminative subtypes, which are shared between data modalities, can be estimated from one data modality and optimized according to the predictions induced in another modality. We apply this approach to lung cancer by training a deep neural network to predict discriminative subtypes from histopathology images, and use these predicted subtypes to generate models which classify adenocarcinoma, squamous cell carcinoma, and healthy tissue based on transcriptomic signatures. In this way, we optimize the latent discriminative subtypes through induced prediction loss, and the discriminative subtypes are interpreted with standard interpretation of transcriptomic predictive models. Our framework achieves state-of-the-art classification accuracy (F1-score of 0.97) and identifies discriminative subtypes which link histopathology images to transcriptomic explanations without requiring pre-specification of morphological patterns or transcriptomic processes.
BibTeX
@misc{lengerich2020discriminative,
  author = {Benjamin J. Lengerich and Maruan Al-Shedivat and Amir Alavi and Jennifer Williams and Sami Labbaki and Eric P. Xing},
  title = {{Discriminative Subtyping of Lung Cancers from Histopathology Images via Contextual Deep Learning}},
  year = {2020},
  doi = {10.1101/2020.06.25.20140053},
  howpublished = {medRxiv},
}
Figure