For years, the field of protein informatics treated proteins primarily as text. By viewing a protein as a linear string of amino acids, researchers successfully adapted powerful natural language processing architectures—such as Transformers, LSTMs, and BERT—to learn the "grammar" of evolution. These protein language models (PLMs) can predict function, identify binding sites, and generate novel peptides solely by analyzing sequence patterns.
However, a fundamental bottleneck limits this approach: proteins do not function as one-dimensional strings. They fold into intricate three-dimensional shapes. When a model relies exclusively on sequence data, it operates under a linear illusion, blind to the spatial reality that governs physical interactions. A growing body of research demonstrates that bridging sequence-level semantics with explicit 3D structural information is essential for truly accurate and robust protein modeling.
The Blind Spots of Sequence-Only Models
To understand why sequence-only models struggle, consider how proteins interact. A binding interface or an active site is rarely a continuous stretch of text. Instead, it is a 3D pocket formed by amino acids that might be hundreds of positions apart in the linear sequence but are brought into close proximity by the folding process.
When models attempt to predict complex biophysical properties from sequences alone, they must infer these spatial relationships implicitly. For instance, the cross-attention model developed by Tingfang Wu et al. in CAPLA: improved prediction of protein-ligand binding affinity by a deep learning approach based on a cross-attention mechanism successfully captures sequence-level interactions between proteins and ligands, yet the authors explicitly note that omitting 3D structural data limits its ultimate accuracy. Similarly, in antibody engineering, the paratope predictor designed by Pan Zhang et al. (DeepANIS: Predicting antibody paratope from concatenated CDR sequences by integrating bidirectional long-short-term memory and transformer neural networks) relies entirely on sequence data, leaving a gap in the model's understanding of the physical shape of the binding interface.
This sequence-only limitation manifests across diverse tasks:
- Functional Annotation: Models like NetGO 3.0 by Yunjia Liu et al. (NetGO 3.0: Protein Language Model Improves Large-scale Functional Annotations) leverage large-scale PLM embeddings to annotate protein functions, but find that sequence embeddings alone cannot fully capture the structural determinants of complex functions.
- Peptide Design: Frameworks like the antimicrobial peptide predictors developed by Wenxuan Xing et al. (iAMP-Attenpred) and Linlin Zhuo et al. (Diff-AMP) miss out on the spatial arrangements and interaction networks that dictate whether a peptide can successfully disrupt a bacterial membrane.
- Targeted Classification: Early machine learning approaches, such as the sequence-based classifiers for specific receptors proposed by Zhijun Liao et al. (In SilicoPrediction of Gamma-Aminobutyric Acid Type-A Receptors Using Novel Machine-Learning-Based SVM and GBDT Approaches) or DNA-binding proteins by Jiyun Zhou et al. (enDNA-Prot: Identification of DNA-Binding Proteins by Applying Ensemble Learning), consistently identify the integration of structural features as the primary avenue for overcoming performance ceilings.
Where Text Fails: The Twilight Zone and Viral Mimicry
The necessity of structural data becomes starkly apparent in two specific biological phenomena: remote homology and molecular mimicry.
In protein evolution, there exists a region known as the "twilight zone"—a state where two proteins share less than 20–30% sequence similarity but fold into virtually identical 3D structures and perform the same function. Sequence-only search tools fail completely here because the "text" has mutated beyond recognition. To address this, the late-interaction retrieval model ProtoCol, designed by Gabrielle Cohn et al., attempts to infer structural relationships from residue-level sequence embeddings. Yet, the authors conclude that explicitly injecting 3D coordinates is the only way to reliably bridge the gap for highly divergent protein pairs.
Sequence A: M-K-V-L-A-A-G-K-T ... (Low Similarity)
Sequence B: A-R-I-F-S-S-D-P-E ...
Linear View (PLM): "These look completely unrelated."
Spatial View (3D): [Both fold into identical alpha-helix bundles] -> Functional Homologs
Conversely, sequence similarity can sometimes be an illusion. Viruses exploit this through "viral mimicry," evolving sequences that look nearly identical to host proteins to evade the immune system. As demonstrated by Dan Ofer et al. and Michal Linial in Protein language models expose viral mimicry and immune escape, PLMs struggle to differentiate between host and viral proteins because they lack the explicit 3D context. Subtle geometric shifts in the physical structure determine whether a protein triggers an immune response or slips past undetected—nuances that sequence-only models cannot see.
The Solution: Multimodal Fusion
To overcome these limitations, the frontier of protein AI focuses on multimodal models that fuse sequence semantics with geometric priors. This is not merely about adding structure as a post-processing step; it is about grounding the language model's representations in physical space during training.
A prime example is the antibody representation model S 2 ALM, developed by Mingze Yin et al.. While previous models focused strictly on sequence data, the authors pre-trained a large language model to simultaneously process amino acid sequences and their 3D coordinates. This dual-view learning ensures that the model's internal representations reflect both evolutionary constraints and physical geometry.
Similarly, in generative tasks like antibody design, purely structural models often suffer from "vocabulary collapse," generating physically plausible but evolutionarily non-viable sequences. To solve this, Mansoor Ahmed et al. introduced EvoStruct: Bridging Evolutionary and Structural Priors for Antibody CDR Design via Protein Language Model Adaptation. By connecting a PLM with an equivariant Graph Neural Network (GNN), the model uses the GNN to enforce geometric constraints while the PLM guides the sequence generation toward natural, diverse amino acid choices.
Why the Synergy Goes Both Ways
Importantly, the relationship between sequence and structure is symbiotic. Just as sequence models need structure to ground them in physical reality, structural models need sequence data to understand evolutionary history.
Purely geometric models, such as the 3D convolutional neural network developed by Guillaume Pagès et al. and Benoit Charmettant (Protein model quality assessment using 3D oriented convolutional neural networks), can evaluate the physical viability of a protein fold, but the authors note that omitting sequence information deprives the model of crucial evolutionary context. Sequence data implicitly contains millions of years of natural selection, highlighting which mutations are tolerated and which are lethal.
By combining evolutionary encodings with structural message passing, frameworks like the one proposed by Zhiqiang Zhong et al. (Efficiently Predicting Mutational Effect on Homologous Proteins by Evolution Encoding) or the early probabilistic backbone models like TorusDBN by Kanti V. Mardia et al. can accurately predict how single-point mutations alter a protein's stability and function.
Ultimately, treating a protein as either a string of text or a static 3D shape captures only half the picture. The most powerful models treat them as both: using sequence data to capture evolutionary semantics, and 3D structural data to enforce the laws of physical geometry.
Key Foundational Work
A generative, probabilistic model of local protein structure (2008) — contributed 1 concept elements to this direction.
Transmembrane Topology and Signal Peptide Prediction Using Dynamic Bayesian Networks (2008) — contributed 1 concept elements to this direction.
SeqRate: sequence-based protein folding type classification and rates prediction (2010) — contributed 1 concept elements to this direction.