Traditional computational biology treats proteins and peptides as strings of text drawn from a strict 20-letter alphabet representing the standard natural amino acids. While this abstraction works well for analyzing natural evolutionary history, it presents a major bottleneck for modern drug discovery. To optimize therapeutic properties like metabolic stability, cell permeability, and binding affinity, researchers must routinely move beyond nature's toolkit by incorporating non-canonical amino acids (ncAAs), post-translational modifications (PTMs), and structural alterations like cyclization (Ruochi Zhang et al. in PepLand: a large-scale pre-trained peptide representation model for a comprehensive landscape of both canonical and non-canonical amino acids; Leyao Wang et al. in PepDoRA: A Unified Peptide Language Model via Weight-Decomposed Low-Rank Adaptation).
When standard computational models encounter these modified sequences, they fail because they lack the vocabulary to represent them. To resolve this, a major research direction has emerged focused on building chemically aware representations that treat peptides not merely as biological sequences, but as flexible chemical structures.
The Breakdown of the Biological Alphabet
Why do standard models fail when a peptide contains a non-canonical residue? The issue lies in tokenization and pretraining constraints.
Most state-of-the-art protein language models (pLMs) are pretrained on massive databases of natural sequences dominated by the 20 standard amino acids. When these models encounter a synthetic modification or an altered residue, they typically map it to an unknown token (<UNK>) or treat it identically to its unmodified wild-type precursor (Peng Fei et al. in PTM-Mamba: a PTM-aware protein language model with bidirectional gated Mamba blocks). This simplification erases the exact chemical changes—such as a single added methyl group or a halogenation—that dictate the molecule's clinical efficacy (Jiahui Zhang et al. in PepBenchmark: A Standardized Benchmark for Peptide Machine Learning).
This representation gap propagates across the entire computational pipeline:
- Descriptor Scarcity: Traditional machine learning pipelines struggle to generalize to synthetic space because they lack standardized descriptors for non-natural variants (Jian Song et al. in Identification of Inhibitors of MMPS Enzymes via a Novel Computational Approach).
- Rigid Interface and Design Rules: Interface analysis models and automated design frameworks are heavily optimized for standard residues, rendering them incapable of evaluating non-protein components or synthetic backbones (Chee Keong Kwoh in Beta Atomic Contacts: Identifying Critical Specific Contacts in Protein Binding Interfaces; Georg Künze in Guiding protein design choices by per-residue energy breakdown analysis with an interactive web application).
- Modeling Limitations: Standard molecular modeling and evolutionary design protocols struggle to accommodate the geometric complexities introduced by ncAAs and cyclization (Clement Agoni et al. in Molecular Modelling in Bioactive Peptide Discovery and Characterisation; Rodrigo Ochoa et al. in PARCE: Protocol for Amino acid Refinement through Computational Evolution).
As a result, traditional models exhibit a stark disconnect between rigid computational rules and empirical biochemical complexity (Junling Wang et al. in Target site specificity and in vivo complexity of the mammalian arginylome). Conversely, general chemical language models understand small-molecule chemistry but fail to capture the long, polymer-like macrostructures and evolutionary context inherent to peptides (Aaron L. Feller et al. in Scaling SMILES-Based Chemical Language Models for Therapeutic Peptide Engineering).
Mechanisms for Chemically Aware Modeling
To solve this dilemma, researchers are developing hybrid architectures that combine the macrostructural awareness of biological models with the atomic granularity of chemical models. Two primary mechanisms drive this transition:
1. Cross-Modal Adaptation and Gating
Instead of training a model from scratch, researchers can inject chemical awareness into existing architectures. One approach uses parameter-efficient fine-tuning, such as Weight-Decomposed Low-Rank Adaptation (DoRA), to adapt a general chemical language model to the peptide domain (Leyao Wang et al. in PepDoRA: A Unified Peptide Language Model via Weight-Decomposed Low-Rank Adaptation). This allows the model to retain its understanding of fundamental chemical bonds while learning the macrostructural rules of long peptide chains.
Alternatively, architectural gating can be used to fuse biological and chemical representations explicitly. In PTM-Mamba, the authors combined bidirectional Mamba blocks with state-of-the-art ESM-2 embeddings through a novel gating mechanism (Peng Fei et al. in PTM-Mamba: a PTM-aware protein language model with bidirectional gated Mamba blocks). The ESM-2 base maintains the evolutionary context of the sequence, while the gated Mamba blocks track the precise locations and chemical identities of modifications, preventing the loss of information caused by standard tokenization.
2. All-Atom String Representations
A more radical shift abandons the amino acid alphabet entirely, representing the entire peptide chain as a continuous chemical graph via strings like SMILES or SELFIES. By breaking the peptide down to its constituent atoms and bonds, any synthetic modification, non-natural residue, or cyclization event is natively supported by the vocabulary (Yinuo Zhang et al. in PeptiVerse: A Unified Platform for Therapeutic Peptide Property Prediction). This allows models like PeptideCLM-2 to scale effectively across chemical space, learning property predictions directly from the full molecular structure (Aaron L. Feller et al. in Scaling SMILES-Based Chemical Language Models for Therapeutic Peptide Engineering).
Unlocking De Novo Design and Property Prediction
By transitioning to chemically aware representations, computational tools can accurately evaluate and generate novel therapeutics without relying on slow, manual wet-lab screening (Marc Oeller et al. in Sequence-based prediction of the intrinsic solubility of peptides containing non-natural amino acids).
Traditional Pipeline:
[Natural Sequence] ──> [pLM (20-Letter Alphabet)] ──> Blind to ncAAs/PTMs
Chemically Aware Pipeline:
[Modified Sequence] ──> [All-Atom / Hybrid Representation] ──> [Property Prediction / Generative Design]
For property prediction, platforms can map the continuous landscape of both canonical and non-canonical variants to predict critical developability traits—such as intrinsic solubility, membrane permeability, and anticancer activity—even when the sequence contains highly exotic synthetic modifications (Marc Oeller et al. in Sequence-based prediction of the intrinsic solubility of peptides containing non-natural amino acids; Joshua Zhi En Tan et al. in Topology-enhanced machine learning model (Top-ML) for anticancer peptide prediction).
For generative tasks, models are no longer constrained to simply shuffling natural templates (Cibele Nicolaski Pedron et al. in Molecular hybridization strategy for tuning bioactive peptide function). Generative frameworks like PepINVENT use transformer-based text-infilling over specialized chemical notations to invent entirely new non-natural building blocks in situ, optimizing the peptide's topology for targeted drug-like properties (Gökçe Geylan et al. in PepINVENT: generative peptide design beyond natural amino acids). Similarly, discrete diffusion models operating on all-atom SELFIES strings can generate structurally viable, highly diverse sequences that natively incorporate non-canonical functional groups (Amelia Villegas-Morcillo et al. in All-Atom Protein Sequence Design using Discrete Diffusion Models).
Ultimately, by shifting from a rigid biological alphabet to a flexible chemical representation, these models unlock the ability to simulate, predict, and design the full chemical diversity required for modern peptide therapeutics.
Key Foundational Work
Identification of Inhibitors of MMPS Enzymes via a Novel Computational Approach (2018) — contributed 1 concept elements to this direction.
Target site specificity and in vivo complexity of the mammalian arginylome (2018) — contributed 1 concept elements to this direction.
Beta Atomic Contacts: Identifying Critical Specific Contacts in Protein Binding Interfaces (2013) — contributed 1 concept elements to this direction.