MiMSI - a deep multiple instance learning framework improves microsatellite instability detection from tumor next-generation sequencing

John Ziegler, Jaclyn F. Hechtman, Ryan Ptashkin, Gowtham Jayakumaran, Sumit Middha, Shweta S. Chavan, Chad Vanderbilt, Deborah F. DeLair, Jacklyn Casanova, Jinru Shia, Nicole DeGroat, Ryma Benayed, Marc Ladanyi, Michael F. Berger, Thomas J. Fuchs, Ahmet ZehirbioRxiv 2020 · DOI

Abstract

Microsatellite instability (MSI) is a critical phenotype of cancer genomes and an FDA-recognized biomarker that can guide treatment with immune checkpoint inhibitors. Recent work has demonstrated that next-generation sequencing data can be used to identify samples with MSI-high phenotype. However, low tumor purity, as frequently observed in routine clinical samples, poses a challenge to the sensitivity of existing algorithms. To overcome this critical issue, we developed MiMSI, an MSI classifier based on deep neural networks and trained using a dataset that included low tumor purity MSI cases in a multiple instance learning framework. On a challenging yet representative set of cases, MiMSI showed higher sensitivity (0.940) and auROC (0.988) than MSISensor (sensitivity: 0.57; auROC: 0.911), an open-source software previously validated for clinical use at our institution using MSK-IMPACT large panel targeted NGS data.

Introduction

Microsatellite instability (MSI) is a genomic signature resulting from a deficient DNA mismatch repair system. Identifying MSI-high tumors is clinically vital, as these patients often show exceptional responses to immunotherapy. While traditional methods like immunohistochemistry (IHC) or polymerase chain reaction (PCR) are standard, there is a growing need to detect MSI directly from Next-Generation Sequencing (NGS) data used for broader genomic profiling.

Current computational tools, such as MSISensor, primarily rely on comparing the statistical distributions of microsatellite lengths between tumor and normal samples. However, these methods often fail in real-world clinical scenarios characterized by low tumor purity or low sequencing coverage. In such cases, the MSI signal is obscured by "noise" from healthy cells, leading to false negatives or indeterminate results. MiMSI was developed to address these limitations using a deep learning-based Multiple Instance Learning (MIL) framework.

Methods

The MiMSI framework shifts the paradigm from simple statistical counting to a learned feature extraction approach. The process involves three main stages:

  1. Vectorization: Raw NGS reads are converted into 3D vectors that encode essential information, including CIGAR strings (representing insertions and deletions), mapping quality, and reference sequences.
  2. Multiple Instance Learning (MIL): Since MSI status is known at the patient level but not necessarily for every individual microsatellite site, the authors employed an MIL strategy. Here, a patient sample is treated as a "bag" containing multiple genomic "instances" (sites). The model learns to identify the collective signature of instability across the bag.
  3. Deep Learning Architecture: The model utilizes a modified ResNet-18 architecture to extract features from each site. These features are aggregated into a sample-level embedding, which is then processed by a final classifier to predict the probability of MSI.
Figure 1: Schematic of the MiMSI architecture, illustrating the conversion of NGS reads to vectors and the Multiple-Instance Learning model.

The model was trained and validated on a high-quality dataset of 458 cases from Memorial Sloan Kettering Cancer Center (MSKCC), with ground truth established via PCR or IHC. This dataset specifically included challenging low-purity and low-coverage cases to ensure robustness.

Results

MiMSI demonstrated superior performance over the clinical standard, MSISensor, particularly in challenging samples. On an independent test set of 137 cases, the MiMSI 200X coverage model achieved a sensitivity of 0.940 and an auROC of 0.987, significantly outperforming MSISensor's sensitivity of 0.571 and auROC of 0.911.

Figure 2: Performance comparison on the test set. (D) ROC curves for MiMSI at various coverages vs. MSISensor. (E) Correlation between MiMSI probability scores and MSISensor scores.

Key Insights

  • Resilience to Low Purity: Dilution experiments showed that MiMSI maintains high sensitivity even at tumor purities as low as 6%, a level where MSISensor typically fails.
  • Clinical Utility: In a large-scale cohort of 44,724 clinical samples, MiMSI provided definitive classifications for many cases that MSISensor labeled as "indeterminate," potentially expanding the pool of patients eligible for immunotherapy.
  • Feature Learning: By learning the complex "texture" of instability rather than just counting deletions, MiMSI proves more resilient to the technical noise inherent in clinical sequencing.

Discussion and Future Work

While MiMSI represents a significant advancement, it currently performs best with "matched normal" samples to distinguish true instability from germline variation. Future iterations may focus on improving "tumor-only" detection. Additionally, the model's confidence is dependent on the number of microsatellite sites covered by the sequencing panel, with performance decreasing if fewer than 500 sites are available.

The success of the MIL approach in MSI detection suggests it could be adapted for other complex genomic phenotypes that are difficult to detect in low-purity samples, such as specific mutational signatures or structural variants.

Summary updated

Abstract

Abstract Microsatellite instability (MSI) is a critical phenotype of cancer genomes and an FDA-recognized biomarker that can guide treatment with immune checkpoint inhibitors. Recent work has demonstrated that next-generation sequencing data can be used to identify samples with MSI-high phenotype. However, low tumor purity, as frequently observed in routine clinical samples, poses a challenge to the sensitivity of existing algorithms. To overcome this critical issue, we developed MiMSI, an MSI classifier based on deep neural networks and trained using a dataset that included low tumor purity MSI cases in a multiple instance learning framework. On a challenging yet representative set of cases, MiMSI showed higher sensitivity (0.940) and auROC (0.988) than MSISensor(sensitivity: 0.57; auROC: 0.911), an open-source software previously validated for clinical use at our institution using MSK-IMPACT large panel targeted NGS data.
BibTeX
@misc{ziegler2020mimsi,
  author = {John Ziegler and Jaclyn F. Hechtman and Ryan Ptashkin and Gowtham Jayakumaran and Sumit Middha and Shweta S. Chavan and Chad Vanderbilt and Deborah F. DeLair and Jacklyn Casanova and Jinru Shia and Nicole DeGroat and Ryma Benayed and Marc Ladanyi and Michael F. Berger and Thomas J. Fuchs and Ahmet Zehir},
  title = {{MiMSI - a deep multiple instance learning framework improves microsatellite instability detection from tumor next-generation sequencing}},
  year = {2020},
  doi = {10.1101/2020.09.16.299925},
  howpublished = {bioRxiv},
}
Figure