Machine Learning Optimization of Candidate Antibodies Yields Highly Diverse Sub-nanomolar Affinity Antibody Libraries

Lin Li, Esther Gupta, John Spaeth, Leslie Shing, Rafael Jaimes, Rajmonda S. Caceres, Tristan Bepler, Matthew WalshbioRxiv 2022 · DOI

TL;DR

This paper introduces an end-to-end machine learning framework that uses language models and Bayesian optimization to design highly diverse antibody libraries with significantly improved binding affinities compared to traditional directed evolution methods. This approach accelerates the discovery of therapeutic antibodies by enabling in-silico exploration of antibody design space and prediction of library performance before experimental validation.

What Problem Does This Paper Address?

The discovery and optimization of therapeutic antibodies, specifically single-chain variable fragments (scFVs), is a critical but time-consuming and expensive process. The sheer vastness of possible antibody sequences makes exhaustive searching infeasible. Current methods, like directed evolution, often yield libraries with limited diversity and may require extensive experimental cycles to identify candidates with sufficient binding affinity and desirable developability characteristics. This paper aims to address this by developing a computational approach that can rapidly design large, diverse libraries of high-affinity scFVs, reducing the time and cost associated with early-stage antibody development.

Methods

The core of this work is an end-to-end machine learning pipeline that integrates several key components:

  • Language Model Training: Pre-trained BERT language models are fine-tuned on large datasets of protein and antibody sequences (Pfam and Observed Antibody Space) to learn biologically relevant features and represent scFv sequences.
  • Sequence-to-Affinity Model Training: These language models are further fine-tuned to predict binding affinities of scFVs, incorporating uncertainty quantification using either an ensemble method or Gaussian Processes (GP).
  • Bayesian Optimization for Design: A Bayesian-based fitness landscape is constructed, mapping scFv sequences to a posterior probability of having better binding affinity than a baseline candidate. Bayesian optimization is then used to explore this landscape and design novel scFv sequences.
  • In Silico Library Evaluation: The framework allows for the evaluation of designed libraries' predicted success and diversity before experimental testing, enabling informed design choices.
  • Experimental Validation: The designed scFv libraries are synthesized and experimentally tested using high-throughput assays to validate the in silico predictions.

The novelty lies in the integration of these components into a cohesive pipeline, particularly the use of Bayesian optimization on an ML-extrapolated fitness landscape and the ability to predict library performance in silico.

Datasets

  • Training Data:
    • Pfam database: Used for pre-training general protein language models.
  • Observed Antibody Space (OAS) database: Used for pre-training antibody-specific language models and for training sequence-to-affinity models. This dataset includes immune repertoires from human subjects.
  • Engineered yeast mating assay data: A dataset of ~17,000 heavy chain and ~26,000 light chain scFv binding measurements against a single target peptide, generated through random mutations of a candidate antibody (Ab-14). This data was used for supervised training of sequence-to-affinity models.

Key Insights and Ideas

  • ML-driven design can outperform directed evolution: The paper demonstrates that ML-designed antibodies can achieve significantly higher binding affinities than those generated by traditional directed evolution methods.
  • Diversity is crucial, but needs balance: While highly diverse libraries are desirable, the study shows that extreme diversity can sometimes lead to reduced affinity if it pushes sequences too far beyond the model's predictive capabilities. The choice of sampling algorithm (e.g., Gibbs sampling for higher diversity) plays a critical role.
  • In silico evaluation is a game-changer: The ability to predict a library's success and diversity before experimental synthesis and testing is a major advantage, allowing for more efficient design cycles and informed decision-making.
  • Language models capture essential antibody features: The use of pre-trained language models effectively distills complex biological information from vast sequence data, enabling better prediction of binding affinities.
  • Bayesian optimization enables exploration of vast sequence space: This approach allows for efficient navigation of the antibody sequence space, identifying promising candidates that might be missed by simpler search strategies.

Key Results

  • 28.8-fold improvement in binding: The best heavy-chain scFv designed by the ML approach showed a 28.8-fold improvement in binding affinity compared to the best scFv from the directed evolution (PSSM) approach.
  • 99% success rate: In the most successful ML-designed library, 99% of the generated scFvs showed improved binding over the initial candidate scFv.
  • High diversity: ML-optimized libraries exhibited significantly higher mutational distances from the original antibody and among themselves compared to the PSSM library, indicating greater diversity. For example, some En-Gibbs library sequences were on average 15.6 mutations away from the original Ab-14-H.
  • Accurate in silico prediction of library performance: The in silico performance metric showed a strong rank correlation (0.8) with actual measured percent of success for both PSSM and ML-generated libraries.
  • Ensemble model superiority: The ensemble sequence-to-affinity model generally outperformed the GP model in predicting affinities, especially for mutationally distant sequences.

Limitations and Future Work

  • Model predictability limits: While the ML models can extrapolate to distant mutants, their predictability decreases with increasing mutational distance, which can impact affinity at extreme levels of diversity.
  • Cost of DNA synthesis: The current method is limited by the cost of synthesizing DNA for large libraries, influencing the choice of random mutation levels for training data.
  • Future work:
    • Investigate combining pre-trained generative language models with this framework to design initial mutagenesis libraries for even higher success rates.
  • Conditioning pre-trained models on target epitope features for more targeted library design.
  • Extending the framework to other protein engineering tasks beyond scFVs.
  • Exploring multi-objective optimization for other biophysical properties (e.g., hydrophobicity, isoelectric point) alongside binding affinity.
  • Optimizing the trade-off between financial resources and the number of measurement cycles for future engineering attempts.

Summary updated

Abstract

Abstract Therapeutic antibodies are an important and rapidly growing drug modality. However, the design and discovery of early-stage antibody therapeutics remain a time and cost-intensive endeavor. In this work, we present an end-to-end Bayesian, language model-based method for designing large and diverse libraries of high-affinity single-chain variable fragments (scFvs). We integrate target-specific binding affinities with information from millions of natural protein sequences in a probabilistic machine learning framework to design thousands of scFvs that are then empirically measured. In a head-to-head comparison with a directed evolution approach, we show that the best scFv generated from our method represents a 28.8-fold improvement in binding over the best scFv from the directed evolution. Additionally, 99% of the designed scFvs in our most successful library are improvements over the initial candidate scFv. By comparing a library’s predicted success to actual measurements, we demonstrate our method’s ability to explore tradeoffs between library success and diversity during the design phase and prior to experimental testing. The results of our work highlight the significant impact machine learning models can have on scFv development. We expect our end-to-end method to be broadly applicable and able to provide value to other protein engineering tasks.
BibTeX
@misc{li2022machine,
  author = {Lin Li and Esther Gupta and John Spaeth and Leslie Shing and Rafael Jaimes and Rajmonda S. Caceres and Tristan Bepler and Matthew Walsh},
  title = {{Machine Learning Optimization of Candidate Antibodies Yields Highly Diverse Sub-nanomolar Affinity Antibody Libraries}},
  year = {2022},
  doi = {10.1101/2022.10.07.502662},
  howpublished = {bioRxiv},
}
Figure