Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unannotated pathology slides

Adalberto Claudio Quiros, Nicolas Coudray, Anna Yeaton, Xinyu Yang, Bojing Liu, Hortense Le, Luis Chiriboga, Afreen Karimkhan, Navneet Narula, David A. Moore, Christopher Y. Park, Harvey I. Pass, André L. Moreira, John Le Quesne, Aristotelis Tsirigos, Ke YuanNature Communications 2024 · DOI

Abstract

Cancer diagnosis and management depend upon the extraction of complex information from microscopy images by pathologists, which requires time-consuming expert interpretation prone to human bias. Supervised deep learning approaches have proven powerful, but are inherently limited by the cost and quality of annotations used for training. Therefore, we present Histomorphological Phenotype Learning, a self-supervised methodology requiring no labels and operating via the automatic discovery of discriminatory features in image tiles. Tiles are grouped into morphologically similar clusters which constitute an atlas of histomorphological phenotypes (HP-Atlas), revealing trajectories from benign to malignant tissue via inflammatory and reactive phenotypes. These clusters have distinct features which can be identified using orthogonal methods, linking histologic, molecular and clinical phenotypes. Applied to lung cancer, we show that they align closely with patient survival, with histopathologically recognised tumor types and growth patterns, and with transcriptomic measures of immunophenotype. These properties are maintained in a multi-cancer study.

Introduction

In modern oncology, the microscopic examination of tissue slides remains the "gold standard" for diagnosis. However, the manual interpretation of these slides is inherently slow, subjective, and susceptible to human bias. While supervised deep learning has emerged as a powerful tool for automating pathology tasks, it faces a significant "annotation bottleneck": the requirement for thousands of high-quality, expert-labeled images. Furthermore, supervised models are often restricted to identifying features already known to human experts, potentially overlooking novel sub-visual patterns of clinical significance.

To address these challenges, we developed Histomorphological Phenotype Learning (HPL). HPL is a self-supervised framework designed to automatically discover cancer patterns in pathology slides without manual labels. By grouping image tiles into an "atlas" of phenotypes, the system can predict patient outcomes and identify molecular signatures across diverse cancer types.

The HPL Framework

The HPL pipeline consists of four integrated stages designed to move from raw whole-slide images (WSIs) to interpretable clinical representations:

  1. Tiling and Pre-processing: WSIs are decomposed into small 224x224 pixel tiles to facilitate high-resolution analysis.
  2. Self-Supervised Learning (SSL): The framework utilizes the Barlow Twins algorithm. Unlike traditional supervised learning, SSL trains the network to recognize features by comparing distorted versions of the same tile, learning representations that are invariant to noise or color variations.
  3. Community Detection: Tile representations are clustered using the Leiden algorithm, a method typically used in social network analysis, to group morphologically similar tiles into Histomorphological Phenotype Clusters (HPCs).
  4. Compositional Analysis: Each patient slide is represented as a compositional vector, reflecting the relative abundance of each discovered phenotype across the tissue.
The HPL framework: from raw slides to self-supervised feature extraction and clustering.

Biological Grounding of Phenotypes

A critical aspect of HPL is that the discovered clusters are not merely mathematical abstractions; they correspond to biological and clinical realities. The resulting HP-Atlas reveals clear trajectories from healthy tissue to malignant tumors, passing through intermediate inflammatory and reactive states.

When reviewed by pathologists, these AI-discovered clusters were found to align with established histopathological features, such as lepidic growth patterns or lymphocytic infiltration. This validation confirms that the model captures meaningful biological information without prior human instruction. Furthermore, in multi-cancer studies, "universal" phenotypes were identified. For example, clusters rich in Tumor-Infiltrating Lymphocytes (TILs) were consistently associated with improved survival across multiple cancer types.

Biological grounding of discovered clusters, showing alignment with tissue types like lymphocytes, stroma, and tumor.

Clinical Performance and Results

The HPL framework demonstrated robust performance across several clinical tasks using datasets from The Cancer Genome Atlas (TCGA) and independent cohorts from NYU:

  • Survival Prediction: In lung adenocarcinoma (LUAD), HPL achieved a mean concordance index (c-index) of 0.65 for overall survival and 0.74 for recurrence-free survival. These results are comparable to state-of-the-art supervised models.
  • Cancer Subtyping: The model successfully distinguished between LUAD and lung squamous cell carcinoma (LUSC) with an AUC of 0.99 on external validation sets.
  • Prognostic Value: HPL-based risk scores outperformed the official International Association for the Study of Lung Cancer (IASLC) grading system in predicting recurrence in certain test scenarios.

Discussion and Future Directions

HPL significantly reduces the reliance on manual labels while providing an interpretable "look under the hood" for pathologists. By representing patients as a mix of visual clusters, the model allows clinicians to see exactly which tissue patterns drive high-risk predictions.

However, the framework still requires expert interpretation to "name" the discovered clusters post-hoc. Additionally, capturing sub-cellular details may require increased computational resources. Future research will focus on integrating HPL with spatial transcriptomics to map gene expression directly onto discovered morphological patterns, creating a more comprehensive map of the cancer landscape.

Summary updated

Abstract

Cancer diagnosis and management depend upon the extraction of complex information from microscopy images by pathologists, which requires time-consuming expert interpretation prone to human bias. Supervised deep learning approaches have proven powerful, but are inherently limited by the cost and quality of annotations used for training. Therefore, we present Histomorphological Phenotype Learning, a self-supervised methodology requiring no labels and operating via the automatic discovery of discriminatory features in image tiles. Tiles are grouped into morphologically similar clusters which constitute an atlas of histomorphological phenotypes (HP-Atlas), revealing trajectories from benign to malignant tissue via inflammatory and reactive phenotypes. These clusters have distinct features which can be identified using orthogonal methods, linking histologic, molecular and clinical phenotypes. Applied to lung cancer, we show that they align closely with patient survival, with histopathologically recognised tumor types and growth patterns, and with transcriptomic measures of immunophenotype. These properties are maintained in a multi-cancer study.
BibTeX
@inproceedings{quiros2024mapping,
  author = {Adalberto Claudio Quiros and Nicolas Coudray and Anna Yeaton and Xinyu Yang and Bojing Liu and Hortense Le and Luis Chiriboga and Afreen Karimkhan and Navneet Narula and David A. Moore and Christopher Y. Park and Harvey I. Pass and André L. Moreira and John Le Quesne and Aristotelis Tsirigos and Ke Yuan},
  title = {{Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unannotated pathology slides}},
  booktitle = {Nature Communications},
  year = {2024},
  doi = {10.1038/s41467-024-48666-7},
}
Figure