Overcoming Representation Limitations in Biological Sequences

65 concepts59 papers1 proposal

To apply machine learning to molecular biology, we must first translate the language of life—DNA, RNA, and proteins—into a format that algorithms understand: vectors of numbers. For years, the standard approach treated these biological sequences like text, using simple encodings such as one-hot vectors, $k$-mer frequencies, or handcrafted physicochemical properties.

However, a biological sequence is not just a string of characters. It is a physical entity that folds into three dimensions, possesses non-linear biochemical dynamics, and carries a rich evolutionary history. Relying on flat, static sequence representations creates a severe information bottleneck.

A growing body of research focuses on breaking through this bottleneck. By moving away from rigid, isolated sequence encodings and toward multi-modal, structural, and context-aware representations, modern models can finally capture the true complexity of biological systems.

The Failure Modes of Flat Encodings

To understand why traditional representations fail, consider how a standard model views a DNA sequence. In a one-hot encoding scheme, every nucleotide is represented as an orthogonal vector. This setup implies that the distance between every base pair is identical and independent of its surroundings.

As Jialiang Sun et al. demonstrate in the context of CRISPR off-target predictions, this assumption fails to capture the non-linear biochemical dependencies between base pairs. The functional impact of a mutation changes dramatically depending on its exact position relative to critical genomic motifs—a nuance that flat encodings obscure.

Similarly, when models rely purely on sequential strings, they struggle to learn the true geometry of biological motifs. In tasks like predicting DNA-protein binding, Yuhang Guo et al. point out that traditional sequence-based architectures frequently learn truncated or incomplete versions of binding patterns. Because the model cannot see how the sequence arranges itself in space, it misses the macro-level structure of the motif.

This limitation extends to proteins as well. Relying on basic statistical features like amino acid frequency prevents models from distinguishing between successful and unsuccessful protein synthesis, as it ignores the broader evolutionary and biophysical context (Hannah-Marie Martiny et al.).

The Hidden Dimensions of the Genetic Code

Even when we look strictly at the sequence level, standard representations often discard vital information due to a misunderstanding of biological redundancy.

For instance, multiple codons can translate into the exact same amino acid. Conventional protein models bypass this by training directly on amino acid sequences, assuming the underlying genetic variation is merely "silent." However, the authors of Codon language embeddings provide strong signals for use in protein engineering demonstrate that this omission discards critical signals. By training a language model (CaLM) directly on codon sequences (cDNA), they captured nuances of translation kinetics and mRNA stability that amino acid representations completely miss, outperforming models with orders of magnitude more parameters.

[DNA/mRNA Codons]  --> Capture Translation Kinetics & Stability
       │
       ▼ (Redundancy/Degeneracy loses information)
[Amino Acids]      --> Misses synonymous mutation impacts
       │
       ▼ (One-hot/Static encoding loses structure)
[Flat Vectors]     --> Misses spatial & non-linear dependencies

Beyond the genetic code itself, how we represent evolutionary history matters. For decades, Position-Specific Scoring Matrices (PSSMs) were the default method for encoding evolutionary profiles. Yet, Xiuzhi Sang et al. show that PSSMs fail to capture the full spectrum of evolutionary relationships. By transitioning to Hidden Markov Model (HMM) profiles, models can extract far more nuanced evolutionary constraints, significantly boosting accuracy in downstream tasks like identifying DNA-binding proteins.

Mapping the Geometry of Interactions

In biology, function is dictated by interaction, and interaction happens in three dimensions. When predicting how two molecules bind—whether it is siRNA targeting mRNA, or an RNA molecule interacting with a small molecule drug—incomplete feature representations severely limit predictive power (Zhijian Huang et al.).

To bridge this gap, researchers are shifting toward graph-based and multi-modal representations that embed spatial geometry directly into the feature map:

  • Graph Neural Networks (GNNs): Instead of treating RNA as a flat string, Rongzhuo Long et al. integrate base-pairing probabilities and protein-interaction contexts into a graph structure, allowing the model to capture the physical reality of siRNA efficacy.
  • Cross-Fusion Attention: When modeling interactions between different molecular types, architectures like the one proposed by Zhijian Huang et al. use attention mechanisms to dynamically fuse sequence and graph features, ensuring that the representation of one molecule is explicitly contextualized by the other.
  • Multi-Resolution Integration: For complex tasks like RNA-binding protein (RBP) interaction and design, Jiyue Jiang et al. demonstrate that combining 1D sequences with explicit 3D structural features is essential to capture the multi-modal nature of molecular binding.

The Shift from Handcrafted Features to Deep Embeddings

Historically, when a sequence representation felt inadequate, the standard solution was feature engineering—manually adding properties like hydrophobicity scales, charge, or pseudo-amino acid compositions.

However, Reyhaneh Esmaielbeiki et al. note that simply piling on more handcrafted features yields diminishing returns. Human-defined metrics are inherently biased and struggle to capture the intricate, non-linear relationships embedded within primary structures.

The modern paradigm solves this by replacing manual feature engineering with self-supervised pre-training. Instead of forcing a model to rely on human-defined rules, transfer learning frameworks leverage massive protein language models to extract deep semantic and evolutionary constraints directly from raw sequences (Xinxin Peng et al.). These self-supervised embeddings can then be fine-tuned for specific tasks, bypassing the limitations of static feature maps and unlocking a far more comprehensive understanding of biological function.

Key Foundational Work

Prediction of coordination number and relative solvent accessibility in proteins (2002) — contributed 1 concept elements to this direction.
An Analysis of Active Learning Strategies for Sequence Labeling Tasks (2008) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_limited_feature_representation_7df2742f,
  title = {limited feature representation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/limited-feature-representation/rdr_7df2742f16894189ad25941d643ce118},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_limited_feature_representation_7df2742f_v1,
  title = {limited feature representation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/limited-feature-representation/rdr_7df2742f16894189ad25941d643ce118/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake