TL;DR
This paper introduces VIRAL, a novel framework that combines protein language models, Bayesian active learning, and biophysics to rapidly identify high-fitness viral variants with minimal experimental data. VIRAL significantly accelerates the detection of concerning variants, outperforming random sampling by up to fivefold, and can predict mutation hotspots years in advance, offering a powerful new tool for pandemic preparedness.
What Problem Does This Paper Address?
The emergence of highly transmissible and immune-evasive viral variants, like those seen with SARS-CoV-2, poses a significant threat to global health. Identifying these "high-fitness" variants early is crucial for effective pandemic response, including vaccine updates and public health interventions. However, experimentally characterizing the fitness of every possible viral variant is practically impossible due to limited resources and the sheer scale of potential mutations. Traditional machine learning approaches also struggle in the early stages of a pandemic when experimental data is scarce. This paper tackles the challenge of efficiently and accurately predicting viral variant fitness in a "few-shot" learning scenario, where only a small amount of experimental data is available.
Methods
The core of the VIRAL framework is its integration of three key components:
- Protein Language Models (PLMs): The authors leverage ESM3, a state-of-the-art PLM that generates sequence embeddings incorporating both sequence and structural information. These embeddings capture the complex relationships between amino acid sequences and their functional properties.
- Bayesian Active Learning with Gaussian Processes (GPs): Active learning is employed to intelligently select which variants to experimentally test. GPs are used to model the relationship between the PLM embeddings and experimental binding affinities (dissociation constants, $K_D$). Crucially, GPs provide uncertainty estimates for their predictions, allowing the active learning strategy to balance exploring known high-fitness regions (exploitation) with investigating uncertain, potentially high-reward regions (exploration).
- Biophysical Model: A pre-trained biophysical model (based on previous work) translates the predicted binding affinities to ACE2 (receptor binding) and antibodies (immune escape) into a measure of viral fitness (infectivity). This model is interpretable, linking predicted fitness to underlying biophysical properties.
The framework operates in an iterative loop: predict fitness and uncertainty for untested variants, select the most promising variants for experimental validation using an acquisition function (like Upper Confidence Bound - UCB), acquire new experimental data, and retrain the GP model.
Datasets
The study utilizes two primary datasets for training and evaluation:
- Combinatorial mutagenesis data: This dataset consists of experimentally measured binding constants ($K_D$) for combinations of mutations in the SARS-CoV-2 Receptor Binding Domain (RBD) against ACE2 and four monoclonal antibodies. This dataset is valuable for evaluating the model's ability to predict fitness for multi-mutant variants.
- Deep mutational scanning (DMS) data: This dataset provides $K_D$ values for single mutations in the SARS-CoV-2 RBD against ACE2 and escape ratios against antibodies. This is used to assess the model's performance in identifying individual mutations and predicting sites under evolutionary pressure.
The authors also reference the GISAID database for analyzing natural viral evolution and identifying "highly mutable" sites.
Key Insights and Ideas
- The Power of Uncertainty: The Upper Confidence Bound (UCB) acquisition strategy, which explicitly considers prediction uncertainty, significantly outperforms a greedy strategy that only focuses on predicted fitness. This highlights the importance of exploring unknown regions of the sequence landscape to find evolutionarily distant but potentially dangerous variants.
- Structure Matters: Incorporating structural information into the protein language model embeddings (ESM3 + Structure) leads to substantially improved predictive accuracy compared to sequence-only embeddings. This underscores the critical role of protein structure in determining functional properties.
- Early Warning System Potential: VIRAL demonstrates a remarkable ability to identify sites that later become "highly mutable" in natural evolution, offering a predictive advantage of up to two years. This suggests its potential as an early warning system for emerging threats.
- Biophysics as a Foundation: The biophysical model provides an interpretable link between molecular properties (binding affinities) and viral fitness, making the predictions biologically meaningful and allowing for early pandemic application when only $K_D$ data is available.
Key Results
- Accelerated Variant Identification: VIRAL accelerates the identification of high-fitness variants by up to fivefold compared to random sampling, requiring experimental characterization of fewer than 1% of possible variants in the combinatorial dataset.
- Predictive Advantage for Mutation Hotspots: The framework effectively identifies sites that become frequently mutated during natural viral evolution, achieving a predictive advantage of up to two years compared to baseline strategies that rely on observed mutations.
- High Enrichment with UCB: The UCB acquisition strategy achieved an enrichment factor (EF) of 5 after 10 rounds of acquisition on the combinatorial dataset, meaning it found five times more top variants than random sampling. This EF increased to 11 when identifying the top 1% of fitness scores.
- Robustness Across Datasets: The model showed strong performance on both combinatorial and DMS datasets, with robust AUC values ranging from 0.62 to 0.78 for identifying highly mutable sites.
Limitations and Future Work
- Biophysical Model Reliance: The accuracy of VIRAL is dependent on the biophysical model's ability to reliably map binding affinities to viral fitness. While this has been validated for SARS-CoV-2, its generalizability to other viruses may require further investigation.
- Epistasis Complexity: While the integrated framework inherently captures some epistatic interactions, the authors suggest future work could further enhance this by integrating explicitly epistasis-aware kernels.
- Data Requirements for Initial Training: While the framework excels in few-shot learning, an initial training set of experimental data is beneficial. The authors note that zero-shot prediction is feasible, but would initially carry higher uncertainty.
- Generalizability to Other Viruses: The study focuses on SARS-CoV-2. Future work could explore the application of VIRAL to other viruses with pandemic potential, adapting the biophysical model and PLM embeddings as needed.
- Real-time Deployment: The authors envision VIRAL as a valuable tool for intermediate outbreak phases, but acknowledge the need for early antibody identification and population-level infectivity data for full deployment.
Summary updated