When building machine learning models for biological sequences like proteins or RNA, a naive first instinct is to treat them like natural language text: encode each amino acid or nucleotide as a discrete token or a one-hot vector. However, this approach discards billions of years of biological optimization. A raw sequence tells us only what a molecule is, not what it could be or what it must preserve to function.
To bridge this gap, a powerful paradigm has emerged: representing biological sequences through the lens of evolutionary conservation. By transforming static sequences into statistical profiles of evolutionary tolerance, models can infer structural constraints, functional interfaces, and mutational landscapes far more effectively than they can from raw sequence data alone.
The Limits of Discrete Tokens
Consider an alanine residue at a specific position in a protein. In a one-hot encoding scheme, this residue is represented identically whether it sits in a flexible, non-functional surface loop or at the heart of a highly optimized enzyme catalytic site.
In reality, these two positions are governed by entirely different biophysical constraints. If the surface loop mutates to a valine or a glycine, the protein likely functions normally. If the catalytic site mutates even slightly, the protein may become non-functional, rendering the organism unviable. Evolution acts as a natural filter, weeding out deleterious mutations over millennia while preserving functional configurations.
To capture this context-dependent constraint, researchers use substitution matrices and evolutionary profiles. For instance, Montemurro et al. utilize the BLOSUM50 matrix in NetTCR-2.0 to encode T-cell receptor sequences. Instead of treating amino acids as orthogonal categories, substitution matrices scale inputs based on the evolutionary likelihood of one residue replacing another, embedding a baseline of physicochemical similarity directly into the input space.
From Sequences to Evolutionary Profiles
While static substitution matrices provide global rules for amino acid similarity, they do not capture position-specific constraints. To achieve this, models transition from single sequences to Position-Specific Scoring Matrices (PSSMs) and Profile Hidden Markov Models (HMMs).
Raw Sequence: M A L T ...
│
[ Multiple Sequence Alignment ]
│
┌─────┴─────┐
▼ ▼
Evolutionary A [ 0.1 0.8 0.0 ] <- High probability of A at position 2
Profile M [ 0.7 0.0 0.1 ] <- High probability of M at position 1
(PSSM / HMM) L [ 0.0 0.1 0.8 ]
...
The mechanism works by taking a target sequence and running an alignment search (such as PSI-BLAST or HMMER) against a massive database of known proteins to find homologs across different species. By stacking these homologous sequences into a Multiple Sequence Alignment (MSA), we can calculate the frequency of each amino acid at every single position.
As Xiong et al. demonstrate in their work on heme binding prediction, these matrices reflect the exact evolutionary pressure exerted on each residue. If a position is 100% conserved as a histidine across fifty diverse species, the model immediately learns that this position is functionally non-negotiable.
To capture even richer evolutionary signals, models increasingly rely on Profile HMMs. As Morehead et al. show with the DIPS-Plus database, Profile HMMs outperform standard alignments because they capture not just character frequencies, but position-specific insertion and deletion probabilities, providing a mathematical signature of structural tolerance.
Why Evolutionary Profiles Outperform Structure
Intuitively, one might assume that three-dimensional structural coordinates (e.g., from X-ray crystallography) would provide the ultimate feature set for predicting protein functions like binding interfaces. Surprisingly, evolutionary profiles often match or exceed the predictive power of explicit 3D geometry.
In a foundational analysis of protein-DNA interactions, Kauffman et al. quantified the information content of various protein features. The authors discovered that sequence-based evolutionary profiles (PSSMs) contain a more informative signal for identifying DNA-binding sites than many structural features.
Why does this happen? A static 3D structure is merely a snapshot of a protein in a single crystallized state. It does not explicitly show the energetic penalties of mutating a site. An evolutionary profile, however, is an empirical record of what mutations have successfully survived the physical constraints of binding over millions of years. Similarly, in the domain of protein-RNA interfaces, Caragea et al. demonstrated that sequence models leveraging PSSMs achieve performance parity with models using explicit spatial coordinates, proving that evolutionary history effectively encodes redundant, high-fidelity structural information.
Downstream Architectures and Representations
Once an evolutionary profile is generated, how do neural networks ingest it? Rather than flattening these matrices and losing spatial context, modern architectures treat them as structured continuous topological spaces.
Spatial and Distance Transformations
Because a PSSM is essentially a 2D grid (Sequence Length × 20 Amino Acids), convolutional neural networks can treat them similarly to images. As Wu et al. demonstrate in Gm-PLoc, processing these matrices with convolutional layers preserves the sequential and evolutionary context simultaneously.
Alternatively, researchers transform the profile into a fixed-length vector that captures sequence-order information. Xu et al. introduced a distance transformation method that calculates the correlation between evolutionary scores separated by specific sequence distances, allowing standard classifiers like Support Vector Machines to exploit non-local evolutionary patterns.
Remote Homology and Top-n-grams
When two proteins share a common ancestor but have diverged so significantly that their sequence identity drops below 20%, standard alignment tools fail. This is the "twilight zone" of bioinformatics.
To detect these distant relationships, Liu et al. developed an approach that extracts the most frequent residues from evolutionary profiles to construct "Top-n-grams." By applying Latent Semantic Analysis to these profile-derived motifs, the authors showed that models can recognize structural similarities and remote homologies even when the raw sequence text appears entirely random.
Generative Modeling over Profiles
The utility of evolutionary profiles extends beyond classification into protein design. Instead of generating discrete sequences token-by-token, modern frameworks model entire protein families as continuous probability distributions over alignment positions.
In ProfileBFN, Gong et al. apply Bayesian Flow Networks directly to MSA profiles. By operating in this continuous profile space rather than on discrete tokens, the generative model inherently respects the co-variation patterns and evolutionary constraints of a protein family, allowing it to steer the design of novel, functionally viable variants.
Summary
By shifting the representational paradigm from discrete sequence strings to continuous evolutionary profiles, machine learning models gain access to a pre-computed map of biological constraints. Whether predicting subcellular localization, mapping mutational effects, or designing synthetic protein families, leveraging the historical record of natural selection remains one of the most data-efficient strategies for biological sequence modeling.
Key Foundational Work
Prediction of coordination number and relative solvent accessibility in proteins (2002) — contributed 2 concept elements to this direction.
Improving the prediction of protein secondary structure in three and eight classes using recurrent neural networks and profiles (2002) — contributed 2 concept elements to this direction.