Adaptive machine learning for protein engineering

Brian L. Hie, Kevin K. YangarXiv 2021 · OpenReview

Abstract

Machine-learning models that learn from data to predict how protein sequence encodes function are emerging as a useful protein engineering tool. However, when using these models to suggest new protein designs, one must deal with the vast combinatorial complexity of protein sequences. Here, we review how to use a sequence-to-function machine-learning surrogate model to select sequences for experimental measurement. First, we discuss how to select sequences through a single round of machine-learning optimization. Then, we discuss sequential optimization, where the goal is to discover optimized sequences and improve the model across multiple rounds of training, optimization, and experimental measurement.

Introduction

Protein engineering represents a search problem of astronomical proportions. For a protein of just 100 amino acids, the number of possible sequences ($20^{100}$) exceeds the number of atoms in the observable universe. Traditional experimental methods, while accurate, are too slow and costly to explore this space effectively. Adaptive machine learning (ML) offers a solution by using surrogate models to guide experimental selection, moving beyond trial-and-error toward an efficient, data-driven discovery process for proteins with optimized properties like stability or catalytic activity.

The Adaptive Machine Learning Framework

The adaptive protein engineering process is composed of four primary elements:

  1. The Property: The target phenotype to be optimized (e.g., fluorescence, binding affinity).
  2. The Predictor (Surrogate Model): A model (ranging from linear regression to deep neural networks) trained to map sequences to properties.
  3. The Prioritizer (Acquisition Function): The logic used to rank and select candidates for testing.
  4. The Proposer (Generative Model): The mechanism that suggests new sequences, either from a fixed design space or via generative models like VAEs or GANs.
The components of adaptive protein engineering and the sequential optimization loop.

These components are often integrated into a Sequential Optimization loop. In this Bayesian optimization framework, the surrogate model is iteratively retrained on new laboratory data, refining its predictions and guiding the next round of experimental measurements.

Acquisition Strategies and the Batching Dilemma

A central challenge in adaptive ML is balancing exploration and exploitation. Greedy acquisition focuses on the top predicted performers, which risks getting trapped in local optima. In contrast, strategies like Upper Confidence Bound (UCB) utilize model uncertainty to encourage exploration of unknown regions of the sequence landscape.

Comparison of acquisition strategies, including greedy, UCB, and batched approaches.

Furthermore, biological experimentation typically occurs in large batches (testing hundreds or thousands of proteins simultaneously). Standard ML optimization often assumes single-sample acquisition. Developing "batched acquisition functions" that ensure diversity within a single round is a critical area of ongoing research to align ML theory with the realities of the laboratory.

Applications and Key Insights

The review synthesizes results from various landmark studies, demonstrating the efficacy of these methods across diverse targets:

  • Efficiency: Neural sequence embeddings combined with simple surrogates can identify high-fitness sequences with minimal training data.
  • Validation: Methods have been successfully validated in "wet labs," discovering improved enzymes and stable proteins that traditional directed evolution might overlook.
  • Uncertainty as a Tool: Utilizing uncertainty through Gaussian Processes or Ensembles allows for systematic exploration, which is essential for finding global optima in complex landscapes.

Challenges and Future Directions

Despite recent progress, several hurdles remain. The "cold start" problem—designing proteins de novo for properties not found in nature—requires the integration of biophysical priors. Additionally, obtaining well-calibrated uncertainty in high-dimensional sequence spaces is difficult. Future work must focus on methods specifically architected for the low-round, high-batch constraints of biological experimentation to fully realize the potential of adaptive ML in protein engineering.

Summary updated

Abstract

Machine-learning models that learn from data to predict how protein sequence encodes function are emerging as a useful protein engineering tool. However, when using these models to suggest new protein designs, one must deal with the vast combinatorial complexity of protein sequences. Here, we review how to use a sequence-to-function machine-learning surrogate model to select sequences for experimental measurement. First, we discuss how to select sequences through a single round of machine-learning optimization. Then, we discuss sequential optimization, where the goal is to discover optimized sequences and improve the model across multiple rounds of training, optimization, and experimental measurement.
BibTeX
@misc{hie2021adaptive,
  author = {Brian L. Hie and Kevin K. Yang},
  title = {{Adaptive machine learning for protein engineering}},
  year = {2021},
  eprint = {2106.05466},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2106.05466},
}
Figure