Structure Aware Protein Representation Learning

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2318 papers357 conceptsfrozen 2026-10-04

For decades, biologists have described proteins as strings of amino acids—a "language of life" where the sequence of letters determines the final product. This analogy fueled the rise of Protein Language Models (pLMs) like RITA by Daniel Hesslow et al., which apply the same scaling laws found in natural language to biological sequences. However, a fundamental gap remains: while a sentence's meaning is largely sequential, a protein’s function is dictated by its three-dimensional shape.

Recent research has shifted toward bridging this gap, moving from models that merely "read" sequences to those that "understand" 3D geometry.

The Sequence Structure Gap

The primary limitation of sequence-only models is their lack of spatial awareness. In a linear sequence, two amino acids might be hundreds of positions apart, yet in a folded protein, they may sit side-by-side, forming a critical active site. As Yang Xiao and colleagues note in their comprehensive survey, this creates non-linear constraints that standard Transformers struggle to capture.

Without explicit 3D information, models often produce "hallucinated" sequences that look plausible on paper but fail to fold into functional shapes in the real world. This is particularly evident in high-precision tasks like antibody design, where Daniel Cutting and Charlotte Deane argue that sequence-based models lack the geometric inductive biases necessary to ensure physical viability.

Injecting Geometric Inductive Biases

To solve this, researchers are "infusing" 3D knowledge into the architecture of language models. The intuition is simple: if the model knows the distance between atoms, it can better predict how they will interact.

One approach, the Protein Structure Transformer (PST) developed by Dexiong Chen and Karsten Borgwardt, integrates lightweight graph neural networks (GNNs) directly into the self-attention blocks of a pretrained pLM. This allows the model to attend to "spatial neighbors" rather than just "sequence neighbors." Similarly, Daniel Peñaherrera and David Ryan Koes developed SI-PLM, which uses structural data during training to guide the model, even if that data isn't available during inference.

From Strings to Graphs and Voxels

If a protein is not a string, what is it? Researchers are exploring two primary alternative representations:

  1. Relational Graphs: GearNet, introduced by Zuobai Zhang and Minghao Xu, treats proteins as 3D graphs where nodes are amino acids and edges represent physical contacts. This allows the model to learn representations directly from the protein's fold, proving more data-efficient than sequence-only pretraining.
  2. 3D Densities (Proxels): Moving away from discrete atoms entirely, Regina Barzilay and Tommi Jaakkola proposed ProxelGen. By representing proteins as voxelized 3D densities, they can treat protein design as a 3D inpainting task, which is far more effective at scaffolding complex functional motifs than linear sequence completion.

Why Geometry Works for Prediction and Design

The shift toward geometry is not just a theoretical preference; it solves specific physical problems. In binding energy prediction, Wengong Jin and Caroline Uhler developed NERE, which uses Euler's rotation equations to model molecular interactions. By treating the protein-ligand complex as a physical system governed by 3D orientation, the model can predict binding affinity in unsupervised settings where sequence-based models fail.

Similarly, for drug discovery, Andrew T. McNutt introduced SPRINT, which uses structure-aware embeddings to screen billions of molecules. By focusing on the 3D geometry of the binding pocket rather than the whole sequence, the model achieves both higher accuracy and extreme computational efficiency.

Ultimately, this research direction suggests that the "language of life" is not written in 1D text, but in the 3D physics of spatial interactions. By grounding language models in geometric reality, we move closer to a future where we can design proteins as precisely as we write code.

Papers in this version

Showing 60 of 318 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 399 papers
  2. Version 1 · 2026-10-04 · 318 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_sequence_structure_gap_sequence_structur_9ce25139,
  title = {sequence-structure gap (sequence-structure information gap)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sequence-structure-gap-sequence-structure-information-gap/rdr_9ce251393c78400684f44305ab2c5975},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_structure_aware_protein_representation_l_9ce25139_v1,
  title = {Structure Aware Protein Representation Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/structure-aware-protein-representation-learning/rdr_9ce251393c78400684f44305ab2c5975/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}