TL;DR
This paper introduces 7-UP, a machine learning framework that can computationally generate high-plex (40+ markers) multiplex immunofluorescence (mIF) data from a much smaller, standard 7-plex panel. By leveraging cellular morphology, 7-UP can predict the expression of many additional biomarkers, enabling richer cellular phenotyping and analysis without the cost and time associated with high-plex imaging.
What Problem Does This Paper Address?
High-plex multiplex immunofluorescence (mIF) techniques like CODEX allow for the simultaneous imaging of over 40 protein biomarkers on a single tissue section, providing incredibly detailed insights into the tissue microenvironment (TME) and disease processes. However, these advanced techniques are expensive and time-consuming to perform, limiting their widespread adoption, especially in clinical settings. In contrast, standard mIF panels typically quantify only 1-7 biomarkers, offering less detailed information. This creates a gap: the desire for high-resolution TME analysis versus the practical limitations of current high-plex technologies. The paper aims to bridge this gap by enabling the generation of high-plex data in silico from readily available low-plex data.
Methods
The core of the 7-UP framework involves a multi-step machine learning pipeline:
- Input Biomarker Selection: A "Concrete autoencoder" is used to identify an optimal set of 7 biomarkers from a larger CODEX panel that are most informative for reconstructing the full panel.
- Morphology Feature Extraction: A ResNet-50 deep learning model, pre-trained on ImageNet, is trained to extract cell shape features and spatial information from image patches of the selected 7 biomarkers. This model learns to represent cell morphology and its neighborhood context.
- Biomarker Imputation: An XGBoost regression model takes the extracted morphological features and the expression values of the 7 input biomarkers to predict the expression of the remaining 30+ biomarkers.
- Downstream Applications: The imputed biomarker expressions are then used for tasks like cell type classification (using a k-nearest neighbors algorithm) and predicting patient outcomes (using a graph neural network).
The novelty lies in the integration of morphological features extracted by a deep learning model to enhance the accuracy of biomarker imputation from a limited set of input markers.
Datasets
The study utilized three CODEX datasets:
- UPMC-HNC Dataset: A primary dataset consisting of 308 samples from 81 patients with head and neck squamous cell carcinomas, used for training and primary evaluation.
- Stanford-HNC Dataset: An external validation dataset with 38 samples from 11 patients with head and neck squamous cell carcinomas, used to assess generalization to the same disease from a different clinical site.
- Stanford-CRC Dataset: An external validation dataset with 292 samples from 161 patients with colorectal cancer, used to assess generalization to a different cancer type.
Key Insights and Ideas
- Morphology is Key: Cellular morphology and its local neighborhood context provide crucial information that significantly improves the accuracy of predicting biomarker expression, beyond just using average expression values.
- Low-Plex Can Unlock High-Plex: A small, carefully selected set of biomarkers (even just 7) can contain enough signal to reconstruct a much richer, high-plex molecular profile of a cell.
- Clinical Utility of Imputed Data: The in silico generated high-plex data is not just an academic exercise; it's accurate enough to be used for clinically relevant tasks like cell type identification and predicting patient survival, comparable to using actual high-plex data.
- Generalizability: The 7-UP framework demonstrates robustness, generalizing well to new cancer types and samples from different clinical sites than those used for training.
Key Results
- Biomarker Imputation Accuracy: 7-UP achieved an average patchwise Pearson correlation coefficient (PCC) of 0.534 for biomarker imputation on the UPMC-HNC dataset when using morphological features. This improved to 0.727 for cell type prediction (patchwise F1 score).
- Cell Type Prediction: The imputed biomarkers allowed for the identification of up to 16 cell types, compared to the 5-7 typically resolved by standard 7-plex panels. The F1 score for cell type prediction was 0.727.
- Patient Phenotype Prediction: Using imputed cell types, the model achieved comparable performance to using ground truth cell types for predicting survival status (AUC 0.841 vs 0.889), recurrence (AUC 0.894 vs 0.887), and HPV status (AUC 0.896 vs 0.929).
- Cross-Site and Cross-Disease Performance: The model maintained stable performance on external validation datasets (Stanford-HNC and Stanford-CRC), indicating good generalization. For example, on the Stanford-CRC dataset, PCC was 0.489 and F1 was 0.614 when using a UPMC-trained model.
Limitations and Future Work
- Inherent Limitations of 7-Plex: While powerful, the imputation is based on a 7-plex panel, meaning some biomarkers may not be as accurately predicted due to inherent limitations of the input data.
- Panel Selection: While the paper presents a method for selecting informative biomarkers, the choice of panel can still influence performance.
- Future Work: The authors suggest that increasing the input panel beyond seven biomarkers could further improve prediction accuracy. They also emphasize the importance of training on diverse disease contexts to ensure generalizability. The framework could also be used to push CODEX systems to analyze even more biomarkers (e.g., 60+).
Summary updated