COVID-19 Vaccine Candidates: Prediction and Validation of 174 SARS-CoV-2 Epitopes

Marek Prachař, Sune Justesen, Daniel Bisgaard Steen-Jensen, Stephan Thorgrimsen, Erik Jurgons, Ole Winther, Frederik Otzen BaggerbioRxiv 2020 · DOI

TL;DR

This paper addresses the critical need for rapid COVID-19 vaccine development by identifying and validating potential T-cell epitopes from SARS-CoV-2. The researchers found that while many prediction tools exist, their accuracy varies significantly, and they often produce a high rate of false positives, highlighting the importance of experimental validation. They present 174 validated SARS-CoV-2 epitopes that stably bind to common HLA types, offering a valuable resource for vaccine design.

What Problem Does This Paper Address?

Developing an effective vaccine against SARS-CoV-2 is a race against time. A key component of a successful vaccine is its ability to elicit a robust T-cell immune response. This response is initiated when T-cells recognize small fragments of viral proteins, called epitopes, presented on the surface of infected cells by Human Leukocyte Antigen (HLA) molecules. SARS-CoV-2 is a large virus, making it computationally infeasible to test every possible peptide fragment for its potential to bind to HLA molecules and trigger an immune response. Therefore, computational tools are used to predict which peptides are most likely to be immunogenic. However, the accuracy and reliability of these prediction tools, especially in the context of a novel virus like SARS-CoV-2, are not fully understood. This paper aims to bridge this gap by evaluating existing prediction tools and experimentally validating their predictions using a novel assay.

Methods

The researchers employed a multi-pronged approach:

  • Peptide Prediction: They used a suite of 15 different HLA-peptide binding prediction tools (including well-established ones like NetMHC and newer ones) to identify potential epitopes from the SARS-CoV-2 proteome. They focused on predicting binding to 11 different HLA allotypes, representing a significant portion of the human population.
  • Experimental Validation (NeoScreen® Assay): To overcome the limitations of purely computational predictions, they utilized a novel in vitro MHC:peptide complex stability assay called NeoScreen®. This assay measures how stably a peptide binds to an HLA molecule under denaturing conditions (using urea). A stable complex is considered more likely to be presented to T-cells and elicit an immune response.
  • Benchmarking and Correlation Analysis: They systematically compared the predictions from the 15 tools against the experimental stability data. This involved calculating Receiver Operating Characteristic (ROC) curves and Area Under the Curve (AUC) values to assess the sensitivity and specificity of each tool. They also used Spearman correlation to evaluate how well the predicted binding rankings matched the experimentally determined stability rankings.
  • Development of a Proof-of-Concept Predictor (PrdX 1.0): To investigate the impact of training data quality, they trained a simple neural network (PrdX 1.0) on their own in-house stability data for a specific HLA allotype (HLA-A*02:01), which is known to have extensive training data available.

Datasets

  • SARS-CoV-2 Proteome: The reference genome sequence of SARS-CoV-2 (assembly MN908947.3) was used as the source for peptide prediction.
  • IEDB (Immune Epitope Database): Previously measured and deposited peptides related to SARS-CoV-2 and their binding affinities were used for comparison and validation.
  • In-house Stability Data: Approximately 2,193 historical in-house stability measurements were used to train the PrdX 1.0 prediction model. This data was a mixture of human cancer-derived and synthetic random peptide measurements.

Key Insights and Ideas

  • Prediction Tools are Imperfect: A major takeaway is that current HLA-peptide prediction tools, while useful for narrowing down candidates, are far from perfect. They exhibit significant variability in performance and a high rate of false positives, meaning many predicted binders do not actually form stable complexes.
  • HLA Allotype Dependence is Crucial: The performance of prediction tools is highly dependent on the specific HLA allotype being considered. This suggests that a "one-size-fits-all" approach to epitope prediction is not effective.
  • Experimental Validation is Essential: The study strongly emphasizes the necessity of in vitro experimental validation (like the NeoScreen® assay) to confirm the immunogenic potential of predicted epitopes. Relying solely on computational predictions can lead to wasted effort and potentially failed vaccine designs.
  • Training Data Quality Matters: The development of PrdX 1.0 demonstrated that a model trained on high-quality, relevant in-house data can outperform general-purpose tools, even for an allotype with abundant public data. This points to potential limitations in the quality and diversity of publicly available training datasets.
  • A Valuable Resource for Vaccine Design: The paper provides a curated list of 174 validated SARS-CoV-2 epitopes, which can significantly accelerate the design and development of COVID-19 vaccines.

Key Results

  • Out of 777 unique peptides predicted to bind to 11 HLA allotypes, 174 peptides were experimentally validated to form stable peptide-HLA complexes.
  • Of these 174 validated epitopes, 48 were previously measured and deposited in the IEDB, while 126 are novel.
  • The study found a high false positive rate among the tested prediction tools, meaning many predicted binders did not show stable binding in the NeoScreen® assay.
  • Performance of prediction tools varied significantly across different HLA allotypes. For example, HLA-A*02:01, despite having the most training data, showed particularly poor performance across most tools.
  • The custom-trained predictor, PrdX 1.0, showed improved performance for HLA-A*02:01 compared to other benchmarked tools.

Limitations and Future Work

  • Limited HLA Coverage: While 11 HLA allotypes were tested, this still represents a fraction of the total human HLA diversity. Future work could expand the validation to a broader range of allotypes.
  • In Vitro vs. In Vivo: The NeoScreen® assay measures peptide-MHC complex stability in vitro. While a strong indicator, it doesn't directly measure T-cell activation in vivo. Further studies would be needed to confirm T-cell responses to these validated epitopes in living organisms.
  • Focus on MHC Class I: The study primarily focused on MHC Class I binding. While MHC Class II plays a role in T-cell immunity, its prediction and validation were less extensive.
  • Data Availability: The authors make their validated epitope data freely available, which is a significant contribution. They suggest this data can be used to improve future prediction models and aid vaccine design. Future work could involve using this dataset to train more robust and accurate prediction algorithms.

Summary updated

Abstract

Abstract The recent outbreak of SARS-CoV-2 (2019-nCoV) virus has highlighted the need for fast and efficacious vaccine development. Stimulation of a proper immune response that leads to protection is highly dependent on presentation of epitopes to circulating T-cells via the HLA complex. SARS-CoV-2 is a large RNA virus and testing of all overlapping peptides in vitro to deconvolute an immune response is not feasible. Therefore HLA-binding prediction tools are often used to narrow down the number of peptides to test. We tested 19 epitope-HLA-binding prediction tools, and using an in vitro peptide MHC stability assay, we assessed 777 peptides that were predicted to be good binders across 11 MHC allotypes. In this investigation of potential SARS-CoV-2 epitopes we found that current prediction tools vary in performance when assessing binding stability, and they are highly dependent on the MHC allotype in question. Designing a COVID-19 vaccine where only a few epitope targets are included is therefore a very challenging task. Here, we present 174 SARS-CoV-2 epitopes with high prediction binding scores, validated to bind stably to 11 HLA allotypes. Our findings may contribute to the design of an efficacious vaccine against COVID-19.
BibTeX
@misc{prachar2020covid,
  author = {Marek Prachař and Sune Justesen and Daniel Bisgaard Steen-Jensen and Stephan Thorgrimsen and Erik Jurgons and Ole Winther and Frederik Otzen Bagger},
  title = {{COVID-19 Vaccine Candidates: Prediction and Validation of 174 SARS-CoV-2 Epitopes}},
  year = {2020},
  doi = {10.1101/2020.03.20.000794},
  howpublished = {bioRxiv},
}
Figure