Sliding Window INteraction Grammar (SWING): a generalized interaction language model for peptide and protein interactions

Alisa A. Omelchenko, Jane C Siwek, Prabal Chhibbar, Sanya Arshad, Iliyan Nazarali, Kiran Nazarali, AnnaElaine Rosengart, Javad Rahimikollu, Jeremy S. Tilstra, Mark J. Shlomchik, David Ryan Koes, Alok V. Joglekar, Jishnu DasbioRxiv 2024 · DOI

TL;DR

SWING (Sliding Window INteraction Grammar) is a novel "interaction language model" that treats the biochemical relationship between two proteins as a unique language. By encoding pairwise residue properties into a grammar, it achieves state-of-the-art performance in predicting peptide-MHC binding and significantly outperforms massive models like AlphaMissense in identifying how specific mutations disrupt protein-protein interactions.

What Problem Does This Paper Address?

While Protein Language Models (pLMs) like ESMFold or AlphaFold have revolutionized our ability to predict individual protein structures, they often stumble when it comes to interactions. Most current methods treat interactions as an afterthought—either by sticking two protein sequences together (concatenation) or by embedding them separately and trying to merge the results later.

These approaches often miss the "fine print" of molecular recognition: the specific, pairwise residue contacts that dictate whether two molecules will bind. Furthermore, existing models struggle to predict "edgetic" variants—mutations that don't necessarily break a protein's overall structure but do disrupt its ability to interact with a specific partner. This is a critical gap in precision medicine, as many diseases are driven by these specific "broken conversations" between proteins.

Methods

The authors introduce SWING, which shifts the focus from the individual sequences to the interaction grammar itself.

  1. Vocabulary Generation: SWING uses a sliding window to compare a peptide (or a protein subsequence) against a target sequence. At each position, it calculates the absolute difference in a biochemical metric (like Grantham polarity or Miyazawa hydrophobicity) between the two residues.
  2. Interaction Language: These differences are rounded and concatenated to form a sequence of "words" (k-mers). This turns a physical interaction into a "document" written in a biochemical language.
  3. Embedding (Doc2Vec): Using a shallow neural network architecture (Doc2Vec), SWING generates low-dimensional embeddings for these interaction documents. Unlike massive Transformers, this architecture is relatively shallow and computationally efficient.
  4. Supervised Prediction: These embeddings are then fed into a classifier (like XGBoost) to predict binding affinity or interaction disruption.

Datasets

The study utilized several high-quality datasets to prove SWING's versatility:

  • pMHC Interactions: Mass Spectrometry (MS) elution data for Human Leukocyte Antigens (HLA) Class I and Class II.
  • De Novo Immunopeptidomes: Newly generated datasets from mouse models of Systemic Lupus Erythematosus (SLE) and Type 1 Diabetes (T1D) to test zero-shot capabilities.
  • Protein-Protein Interactions (PPIs): A curated dataset of over 3,000 missense mutations tested via yeast-2-hybrid assays, including both Mendelian disease-associated variants and common population variants.
  • Benchmarks: Comparisons were made against established tools like NetMHCpan, MixMHCpred, and SOTA variant effect predictors (VEPs) including AlphaMissense, ESM1b, and EVE.

Key Insights and Ideas

  • Interaction as a First-Class Citizen: The core "aha" moment is that the relationship between residues has its own learnable grammar that is distinct from the grammar of individual protein folding.
  • Zero-Shot Generalization: Remarkably, a SWING model trained only on MHC Class I data (which binds short 8-12 AA peptides) was able to accurately predict MHC Class II interactions (which bind longer 13-25 AA peptides). This suggests the model is learning fundamental "binding physics" rather than just memorizing sequence motifs.
  • Edgotypes vs. Pathogenicity: Large-scale VEPs like AlphaMissense are excellent at predicting if a mutation is "pathogenic" (generally bad for the organism), but they are surprisingly poor at predicting if a mutation disrupts a specific interaction. SWING fills this gap, correctly identifying "edgetic" disruptions that SOTA models labeled as benign.
  • Cross-Species Transfer: Models trained on human data transferred effectively to mouse models, highlighting the generalizability of the learned biochemical grammar.

Key Results

  • pMHC Prediction: SWING achieved high accuracy across classes (AUC ~0.72-0.84 for Class I; AUC ~0.90-0.95 for Class II).
  • Superiority in PPI Disruption: In predicting the impact of missense mutations on PPIs, SWING (AUC 0.88) significantly outperformed AlphaMissense (AUC 0.58) and ESM1b (AUC 0.63).
  • Discovery in Data-Sparse Contexts: SWING accurately predicted the immunopeptidome of disease-relevant mouse models (SLE and T1D) even when those specific alleles were entirely absent from the training set.

Limitations and Future Work

While powerful, SWING is not yet a "foundation model" for all biological interactions; it currently requires customization and training for specific biological domains (e.g., pMHC vs. PPI). The authors also note that while the model is robust to different biochemical scales, choosing the most relevant metric for a given context remains an important step.

Future directions include extending SWING to model RNA-protein and DNA-protein interactions, as well as incorporating post-translational modifications (like phosphorylation) into the interaction grammar to capture even more nuanced cellular signaling.

Summary updated

Abstract

The explosion of sequence data has allowed the rapid growth of protein language models (pLMs). pLMs have now been employed in many frameworks including variant-effect and peptide-specificity prediction. Traditionally, for protein-protein or peptide-protein interactions (PPIs), corresponding sequences are either co-embedded followed by post-hoc integration or the sequences are concatenated prior to embedding. Interestingly, no method utilizes a language representation of the interaction itself. We developed an interaction LM (iLM), which uses a novel language to represent interactions between protein/peptide sequences. Sliding Window Interaction Grammar (SWING) leverages differences in amino acid properties to generate an interaction vocabulary. This vocabulary is the input into a LM followed by a supervised prediction step where the LM's representations are used as features. SWING was first applied to predicting peptide:MHC (pMHC) interactions. SWING was not only successful at generating Class I and Class II models that have comparable prediction to state-of-the-art approaches, but the unique Mixed Class model was also successful at jointly predicting both classes. Further, the SWING model trained only on Class I alleles was predictive for Class II, a complex prediction task not attempted by any existing approach. For de novo data, using only Class I or Class II data, SWING also accurately predicted Class II pMHC interactions in murine models of SLE (MRL/lpr model) and T1D (NOD model), that were validated experimentally. To further evaluate SWING's generalizability, we tested its ability to predict the disruption of specific protein-protein interactions by missense mutations. Although modern methods like AlphaMissense and ESM1b can predict interfaces and variant effects/pathogenicity per mutation, they are unable to predict interaction-specific disruptions. SWING was successful at accurately predicting the impact of both Mendelian mutations and population variants on PPIs. This is the first generalizable approach that can accurately predict interaction-specific disruptions by missense mutations with only sequence information. Overall, SWING is a first-in-class generalizable zero-shot iLM that learns the language of PPIs.
BibTeX
@misc{omelchenko2024sliding,
  author = {Alisa A. Omelchenko and Jane C Siwek and Prabal Chhibbar and Sanya Arshad and Iliyan Nazarali and Kiran Nazarali and AnnaElaine Rosengart and Javad Rahimikollu and Jeremy S. Tilstra and Mark J. Shlomchik and David Ryan Koes and Alok V. Joglekar and Jishnu Das},
  title = {{Sliding Window INteraction Grammar (SWING): a generalized interaction language model for peptide and protein interactions}},
  year = {2024},
  doi = {10.1101/2024.05.01.592062},
  howpublished = {bioRxiv},
}
Figure