Integrating protein sequence embeddings with structure via graph-based deep learning for single-residue property prediction

Kevin Michalewicz, Mauricio Barahona, Barbara BraviarXiv 2025 · DOI · arXiv

Abstract

Understanding the intertwined contributions of amino acid sequence and spatial structure is essential to explain protein behaviour. Here, we introduce INFUSSE (Integrated Network Framework Unifying Structure and Sequence Embeddings), a deep learning framework for the prediction of single-residue properties that combines fine-tuning of sequence embeddings derived from a Large Language Model with the inclusion of graph-based representations of protein structures via a diffusive Graph Convolutional Network. To illustrate the benefits of jointly leveraging sequence and structure, we apply INFUSSE to the prediction of B-factors in antibodies, a residue property that reflects the local flexibility shaped by biochemical and structural constraints in these highly variable and dynamic proteins. Using a dataset of 1510 antibody and antibody-antigen complexes from the database SAbDab, we show that INFUSSE improves performance over current machine learning (ML) methods based on sequence or structure alone, and allows for the systematic disentanglement of sequence and structure contributions to the performance. Our results show that adding structural information via geometric graphs enhances predictions especially for intrinsically disordered regions, protein-protein interaction sites, and highly variable amino acid positions -- all key structural features for antibody function which are not well captured by purely sequence-based ML descriptions.

Introduction

Predicting the behavior of individual amino acids (residues) within a protein is a fundamental challenge in structural biology and drug discovery. Recent advances in protein Large Language Models (LLMs) have enabled a deep understanding of the "grammar" of amino acid sequences. However, sequence information alone is often insufficient to capture the full physical behavior of a residue, which is significantly influenced by its three-dimensional (3D) spatial environment.

This limitation is particularly evident in antibodies, which feature hypervariable loops known as Complementary Determining Regions (CDRs) and intrinsically disordered regions. These areas exhibit high evolutionary variability, making it difficult for sequence-only models to generalize effectively. To address this, we present INFUSSE, a framework designed to integrate the evolutionary context of sequences with the physical constraints of 3D geometry.

Methods

The INFUSSE (Integrated Network Framework Unifying Structure and Sequence Embeddings) architecture employs a dual-track approach to process protein data:

  1. Sequence Block ($S_{block}$): High-dimensional embeddings are generated using a frozen ProtBERT model. These are combined with one-hot encodings and processed through learnable non-linear transformations to capture the biochemical context of the sequence.
  2. Graph Block ($G_{block}$): The protein structure is represented as a graph where residues ($\alpha$-carbons) serve as nodes and edges denote spatial proximity. A diffusive Graph Convolutional Network (diff-GCN) utilizes a graph Laplacian and a learnable scale parameter ($t$) to model the propagation of physical constraints throughout the protein structure.
The INFUSSE architecture integrates sequence-based embeddings from ProtBERT with structural information processed through a diffusive Graph Convolutional Network.

The model was trained and evaluated on a dataset of 1,510 high-quality antibody structures from the Structural Antibody Database (SAbDab), including both antibody-antigen complexes and unbound antibodies. Data was filtered for high resolution (< 2.5Å) and sequence identity to ensure robust generalization.

Results and Discussion

INFUSSE achieved state-of-the-art performance in predicting B-factors, a measure of residue flexibility, with a Pearson Correlation Coefficient of R = 0.71. Ablation studies confirmed the value of the integrated approach: the sequence-only model achieved $R = 0.64$, while the structure-only model reached $R = 0.55$. Notably, INFUSSE significantly outperformed a general-purpose LSTM model ($R = 0.48$), demonstrating its superior ability to handle antibody-specific variability.

A key insight from this work is the role of structural information in "high-entropy" regions—positions where the amino acid sequence is highly variable. In these regions, sequence-based LLMs often struggle due to a lack of consistent evolutionary patterns. However, the 3D graph provides the necessary physical "grounding," leading to substantial performance gains. Conversely, for stable, regular motifs like $\beta$-strands in the antibody framework, the sequence-only model performs well, as the sequence pattern is strongly coupled to the fold.

Performance analysis of INFUSSE: (B) Scatter plot showing the correlation between true and predicted B-factors (R=0.71). (C) Box plots of Δgraph demonstrating that structural information provides significant predictive gains in both high and low entropy regions.

The model also showed enhanced sensitivity at binding sites (paratope and epitope), which are critical for understanding immune responses and drug efficacy. To quantify the structural contribution, we introduced $\Delta_{graph}$, a metric representing the improvement in prediction accuracy when structural data is included.

Conclusion

INFUSSE demonstrates that integrating 3D structural graphs with sequence embeddings is essential for accurately modeling the properties of highly variable and disordered protein regions. While the current model uses static PDB snapshots, future work could incorporate dynamic information from Molecular Dynamics simulations to better represent flexible loops. The general-purpose nature of the INFUSSE framework also opens avenues for predicting other single-residue properties, such as epitope locations or chemical reactivity.

Summary updated

Abstract

Understanding the intertwined contributions of amino acid sequence and spatial structure is essential to explain protein behaviour. Here, we introduce INFUSSE (Integrated Network Framework Unifying Structure and Sequence Embeddings), a deep learning framework for the prediction of single-residue properties that combines fine-tuning of sequence embeddings derived from a Large Language Model with the inclusion of graph-based representations of protein structures via a diffusive Graph Convolutional Network. To illustrate the benefits of jointly leveraging sequence and structure, we apply INFUSSE to the prediction of B-factors in antibodies, a residue property that reflects the local flexibility shaped by biochemical and structural constraints in these highly variable and dynamic proteins. Using a dataset of 1510 antibody and antibody-antigen complexes from the database SAbDab, we show that INFUSSE improves performance over current machine learning (ML) methods based on sequence or structure alone, and allows for the systematic disentanglement of sequence and structure contributions to the performance. Our results show that adding structural information via geometric graphs enhances predictions especially for intrinsically disordered regions, protein-protein interaction sites, and highly variable amino acid positions -- all key structural features for antibody function which are not well captured by purely sequence-based ML descriptions.
BibTeX
@misc{michalewicz2025integrating,
  author = {Kevin Michalewicz and Mauricio Barahona and Barbara Bravi},
  title = {{Integrating protein sequence embeddings with structure via graph-based deep learning for single-residue property prediction}},
  year = {2025},
  eprint = {2502.17294},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.17294},
  howpublished = {arXiv},
}
Figure