Enzyme Activity Prediction of Sequence Variants on Novel Substrates using Improved Substrate Encodings and Convolutional Pooling

Zhiqing Xu, Jinghao Wu, Yun S. Song, Radhakrishnan Mahadevan

Abstract

Protein engineering is currently being revolutionized by deep learning applications, especially through natural language processing (NLP) techniques. It has been shown that state-of-the-art self-supervised language models trained on entire protein databases capture hidden contextual and structural information in amino acid sequences and are capable of improving sequence-to-function predictions. Yet, recent studies have reported that current compound-protein modeling approaches perform poorly on learning interactions between enzymes and substrates of interest within one protein family. We attribute this to low-grade substrate encoding methods and overcompressed sequence representations received by downstream predictive models. In this study, we propose a new substrate-encoding based on Extended Connectivity Fingerprints (ECFPs) and a convolutional-pooling of the sequence embeddings. Through testing on an activity profiling dataset of haloalkanoate dehalogenase superfamily that measures activities of 218 phosphatases against 168 substrates, we show substantial improvements in predictive performances of compound-protein interaction modeling. In addition, we also test the workflow on three other datasets from the halogenase, kinase and aminotransferase families and show that our pipeline achieves good performance on these datasets as well. We further demonstrate the utility of this downstream model architecture by showing that it achieves good performance with six different protein embeddings, including ESM-1b, TAPE, ProtBert, ProtAlbert, ProtT5, and ProtXLNet. This study provides a new workflow for activity prediction on novel substrates that can be used to engineer new enzymes for sustainability applications.

Introduction

Predicting how a specific enzyme variant will interact with a new chemical substrate is a "holy grail" in protein engineering. While large-scale language models (LLMs) like ESM-1b have revolutionized our ability to understand protein sequences, current models often stumble when tasked with predicting fine-grained interactions within a single protein family. This limitation is particularly evident in compound-protein interaction (CPI) modeling, where models struggle to distinguish between similar variants acting on diverse substrates.

Xu et al. identify two major technical gaps in current approaches:

  1. Information Loss in Proteins: Most models take high-dimensional protein embeddings and simply average them across the sequence length. This "average pooling" destroys local structural context captured by the upstream LLM.
  2. Low-Resolution Chemistry: Standard molecular representations, such as Morgan bit vectors, often discard intermediate structural information, making it difficult for a model to distinguish between similar but distinct chemical environments.

Methodology

1. Count-Encoding of ECFPs

To address the limitations of binary bit vectors, the researchers introduced count-encoding of Extended-Connectivity Fingerprints (ECFPs). Unlike binary vectors that only record the presence of a substructure, count-encoding captures the frequency of specific chemical substructures. This provides a richer "signature" of the substrate, which is often more predictive of binding affinity and catalytic activity.

2. Convolutional Pooling

Rather than squashing a protein sequence into a single average vector, the authors utilized 1D Convolutional Neural Networks (CNNs) to pool the embeddings. This architecture employs a dual-detector system: a basal detector to capture local amino acid patterns and a superior detector with residual connections to formulate global patterns from a "panoramic view" of the sequence.

Comparison between standard average pooling and the proposed convolutional pooling approach.

3. Integrated Architecture

The final architecture concatenates these rich protein features with the substrate count-vectors, feeding them into a fully connected network for regression or classification. This approach ensures that the downstream model receives high-resolution information from both the protein and the substrate.

The improved prediction architecture featuring 1D convolutions and ECFP6 count encoding.

Experimental Results

The model was rigorously tested on four high-quality enzyme activity datasets, including the Phosphatase Superfamily (~35,000 pairs), Halogenase, Kinase, and Aminotransferase families. The researchers evaluated the model on "novel substrate" and "novel sequence" tasks to test generalization capabilities.

Key Findings

  • Performance Gains: On the phosphatase dataset, the proposed pipeline achieved a Spearman correlation ($R$) of 0.816, a significant improvement over the 0.728 achieved by the baseline model.
  • Generalization: In novel substrate characterization, ECFP6 count-encoding outperformed standard Morgan bit vectors ($R = 0.681$ vs. $0.658$).
  • LLM Robustness: The improvements were consistent across six different protein language models (including ESM-1b, ProtBert, and ProtT5). The best-performing combination used the ProtAlbert embedding with convolutional pooling ($R = 0.820$).

Discussion and Conclusion

This study demonstrates that global average pooling acts as a massive information bottleneck in protein modeling. By replacing it with convolutional pooling, models can better "attend" to specific local motifs critical for enzyme active sites. Furthermore, the results suggest that the density of functional groups (captured by count-encoding) is a vital feature for molecular modeling.

While the model showed some limitations with very small datasets (e.g., Aminotransferases), the overall workflow represents a significant step forward. This architecture is poised to assist in engineering new enzymes for industrial sustainability, such as plastic degradation and carbon capture, by providing more accurate predictions for novel enzyme-substrate pairs.

Summary updated

Abstract

Protein engineering is currently being revolutionized by deep learning applications, especially through natural language processing (NLP) techniques. It has been shown that state-of-the-art self-supervised language models trained on entire protein databases capture hidden contextual and structural information in amino acid sequences and are capable of improving sequence-to-function predictions. Yet, recent studies have reported that current compound-protein modeling approaches perform poorly on learning interactions between enzymes and substrates of interest within one protein family. We attribute this to low-grade substrate encoding methods and overcompressed sequence representations received by downstream predictive models. In this study, we propose a new substrate-encoding based on Extended Connectivity Fingerprints (ECFPs) and a convolutional-pooling of the sequence embeddings. Through testing on an activity profiling dataset of haloalkanoate dehalogenase superfamily that measures activities of 218 phosphatases against 168 substrates, we show substantial improvements in predictive performances of compound-protein interaction modeling. In addition, we also test the workflow on three other datasets from the halogenase, kinase and aminotransferase families and show that our pipeline achieves good performance on these datasets as well. We further demonstrate the utility of this downstream model architecture by showing that it achieves good performance with six different protein embeddings, including ESM-1b, TAPE, ProtBert, ProtAlbert, ProtT5, and ProtXLNet. This study provides a new workflow for activity prediction on novel substrates that can be used to engineer new enzymes for sustainability applications.
BibTeX
@inproceedings{xu2021enzyme,
  author = {Zhiqing Xu and Jinghao Wu and Yun S. Song and Radhakrishnan Mahadevan},
  title = {{Enzyme Activity Prediction of Sequence Variants on Novel Substrates using Improved Substrate Encodings and Convolutional Pooling}},
  booktitle = {MLBC},
  year = {2021},
  url = {https://openreview.net/forum?id=OKI8u3p5S7},
}
Figure