Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Marta Ligero, Garazi Serna, Omar S. M. El Nahhas, Irene Sansano, Siarhei Mauchanski, Cristina Viaplana, Julien Caldéraro, Rodrigo A. Toledo, Rodrigo Dienstmann, R. Vanguri, Jennifer L. Sauter, Francisco Sánchez-Vega, Sohrab P. Shah, Santiago Ramón y Cajal, Elena Garralda, Paolo Nucíforo, Raquel Pérez-López, Jakob Nikolas KatherCancer 2023 · DOI

TL;DR

This paper introduces a weakly supervised deep learning model that predicts PD-L1 expression directly from raw immunohistochemistry (IHC) images of solid tumors. This approach bypasses the need for manual annotation or complex image processing steps, offering a more reproducible and potentially more accurate method for stratifying patients for immunotherapy. The model not only predicts PD-L1 status but also demonstrates its utility in predicting patient response to immune checkpoint inhibitors (ICIs).

What Problem Does This Paper Address?

Immunotherapy, particularly using immune checkpoint inhibitors (ICIs), has revolutionized cancer treatment. However, a key challenge is identifying which patients will benefit from these therapies. Programmed death-ligand 1 (PD-L1) expression on tumor cells is a crucial biomarker for predicting ICI response. Currently, PD-L1 status is assessed by pathologists through manual evaluation of IHC slides. This manual process is subjective, prone to inter-observer variability, and can be time-consuming. While automated image analysis tools exist, many mimic the manual workflow, involving multiple steps like cell detection and quantification, which can be sensitive to artifacts and variations in sample preparation. This leads to limitations in reproducibility and predictive performance. The paper aims to address this by developing a more robust and efficient method for PD-L1 assessment that can better stratify patients for immunotherapy.

Methods

The core of this research is a weakly supervised deep learning (DL) model that operates in an end-to-end fashion. This means it learns to predict PD-L1 status directly from the raw, digitized whole-slide images (WSIs) of PD-L1 stained tissue, without requiring explicit intermediate steps like cell segmentation or manual annotation.

The model employs an attention-based multiple instance learning (MIL) architecture. Here's a breakdown:

  1. Feature Extraction: For each tile (a small image patch) of the WSI, a feature vector is extracted using a pre-trained ResNet50 model (RetCCL). This model was trained on a large dataset of H&E-stained histology slides.
  2. Attention Mechanism: The MIL model assigns a trainable "attention" weight to each tile's feature vector. This allows the model to learn which parts of the image are most important for predicting PD-L1 status.
  3. Aggregation and Classification: The attention weights are used to aggregate the tile-level features into a patient-level representation. This aggregated representation is then fed into a classification head to predict the PD-L1 status (high or low).

This "weakly supervised" approach is key because it doesn't need precise labels for every pixel or cell; it only needs the overall PD-L1 status for the entire slide. The model was trained on PD-L1 IHC slides and then validated on independent cohorts.

Datasets

  • NSCLC-MSK Cohort (Training): This cohort consisted of 233 patients with non-small cell lung cancer (NSCLC) whose PD-L1 stained IHC slides were available, along with their PD-L1 tumor proportion score (TPS) assessed by a pathologist. These patients were treated with immune checkpoint inhibitors (ICIs).
  • Pan-cancer-VHIO Cohort (Validation): This was an external validation set comprising 108 patients with various solid tumor types (including NSCLC, gastrointestinal cancer, melanoma, etc.) treated with ICIs. For this cohort, PD-L1 TPS was assessed, and in some cases, the combined positive score (CPS) was also available.

Key Insights and Ideas

  • End-to-End Weak Supervision: The model's ability to learn PD-L1 status directly from raw WSI data, without explicit cell detection or segmentation, is a significant advancement. This bypasses a major bottleneck in traditional computational pathology.
  • Morphological Integration: The explainability analysis revealed that the model doesn't just focus on brown pigment intensity (PD-L1 staining) but also considers morphological features like lymphocyte conglomerates, suggesting a more holistic understanding of the tumor microenvironment.
  • Generalizability: The model, trained on NSCLC data, demonstrated good performance in a diverse pan-cancer validation cohort, indicating its potential for broader application beyond a single tumor type.
  • Predictive Power for Response: Crucially, the model's predicted PD-L1 status showed a stronger association with progression-free survival (PFS) from ICI treatment than traditional TPS or CPS scores in the validation cohort.

Key Results

  • PD-L1 Classification Performance: The DL model achieved high performance in predicting PD-L1 status (TPS > 1%) with cross-validated mean AUCs of 0.88 ± 0.06 on the NSCLC-MSK training cohort and 0.80 ± 0.03 on the pan-cancer-VHIO validation cohort.
  • Immunotherapy Response Prediction: The predicted PD-L1 status was significantly associated with response to ICIs in terms of PFS. The hazard ratio (HR) for the predicted PD-L1 status was 1.5 (95% CI: 1-2.3, P = 0.049) in the pan-cancer-VHIO cohort, outperforming TPS (HR: 1.4, P = 0.082) and CPS (HR: 1.2, P = 0.386).
  • Qualitative Findings: Explainability maps showed the model focusing on tumor cells and differentiating them from surrounding tissue, including stromal components.

Limitations and Future Work

  • Cohort Size: The authors acknowledge that the training and validation cohorts, while substantial, are relatively small, which may limit the model's generalizability.
  • Histological Diversity: The model showed weaker performance in certain tumor histologies (e.g., melanoma, gastrointestinal, sarcoma) compared to NSCLC, likely due to differences in biology and histology.
  • Staining Protocols and Artifacts: Variations in staining protocols and the presence of artifacts (e.g., melanin in melanoma, necrosis) can influence model performance.
  • H&E vs. IHC Features: While the model uses features trained on H&E images, applying them to IHC might be a limitation.
  • Future Directions: The authors suggest further research on larger and more diverse cohorts to improve generalization. They also propose integrating uncertainty modeling to account for inherent variability in ground truth and exploring the use of CPS as a response biomarker in more advanced patients. The potential impact of interobserver variability in model development is also noted as an area for consideration.

Summary updated

Abstract

The weakly supervised DL model to predict PD-L1 status from raw IHC data, integrating tumor staining intensity and morphology, enables enhanced patient stratification in cancer immunotherapy compared with traditional pathologist assessment.
BibTeX
@inproceedings{ligero2023weakly,
  author = {Marta Ligero and Garazi Serna and Omar S. M. El Nahhas and Irene Sansano and Siarhei Mauchanski and Cristina Viaplana and Julien Caldéraro and Rodrigo A. Toledo and Rodrigo Dienstmann and R. Vanguri and Jennifer L. Sauter and Francisco Sánchez-Vega and Sohrab P. Shah and Santiago Ramón y Cajal and Elena Garralda and Paolo Nucíforo and Raquel Pérez-López and Jakob Nikolas Kather},
  title = {{Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression}},
  booktitle = {Cancer},
  year = {2023},
  doi = {10.1158/2767-9764.crc-23-0287},
}
Figure