Fractal Language Modelling by Universal Sequence Maps (USM)

Jonas S. Almeida, Daniel E. Russ, Susana Vinga, Ines Duarte, Lee Mason, Praphulla M. S. Bhawsar, Aaron Ge, Arlindo L. Oliveira, Jeya Balaji BalasubramanianarXiv 2025 · DOI · arXiv

Abstract

This paper introduces Universal Sequence Maps (USM), a robust numerical encoding for symbolic sequences such as DNA that revitalizes Chaos Game Representation (CGR). By implementing a novel bidirectional convergence approach, USM resolves long-standing seeding biases, providing a scale-free, alignment-free, and perfectly bijective representation of genomic data. The method is computationally efficient and applicable to alphabets of any cardinality, offering a minimalist yet mathematically perfect encoding for sequence comparison and machine learning.

Introduction

In the era of Large Language Models (LLMs), researchers are seeking efficient ways to numerically represent symbolic sequences, such as the alphabets of DNA or proteins. Traditional methods often struggle with two primary challenges: retaining contextual information across multiple scales and managing "edge effects." Earlier fractal encoding methods, like CGR, utilized a static starting point (a "seed") in the center of a coordinate plane. For short sequences or the termini of long ones, this created "tail effects" where the numerical position failed to accurately reflect sequence identity.

Universal Sequence Maps (USM) address these limitations by creating a scale-free representation that allows for sequence comparison without computationally expensive alignment.

Methods

USM builds upon the Chaos Game Representation through an iterated map process. In a square where each corner represents a nucleotide (A, C, G, T), a point moves halfway toward the corner corresponding to the first symbol in a sequence. This process is repeated for every subsequent symbol.

USM forward coordinates for a sample sequence showing the iterated map process.

Bidirectional Encoding and Novel Seeding

To resolve the edge effect problem, the authors introduced Circular and Bidirectional seeding. Instead of a random center point, the tail of the forward pass seeds the head of the backward pass, and vice versa. This causes numerical coordinates to converge to a steady state, effectively treating the sequence as a continuous loop or a Möbius strip.

Schematic of circular seeding, where forward and backward coordinates are linked to ensure convergence.

Distance Metrics

The system utilizes the Chebyshev distance ($Sn$) to calculate the length of matching segments between sequences using only numerical coordinates. This approach is positionally agnostic and highly parallelizable, as calculating a distance matrix does not require knowledge of neighboring cells.

Key Insights

  • Scale-Free Representation: Unlike fixed-length k-mer counting, USM coordinates contain sequence context at all lengths simultaneously.
  • Bijectivity: The mapping is one-to-one, allowing for perfect decoding of numerical coordinates back into the original symbolic sequence.
  • Fractal Geometry as Memory: The recursive folding of the unit square into quadrants acts as a natural memory of the sequence's history.

Results

The approach was validated using the EGFR gene (244,589 nucleotides) and protein sequences. Key performance metrics include:

  • High Efficiency: A quarter-million nucleotides were indexed in 300 milliseconds on a standard laptop.
  • Bias Resolution: Bidirectional seeding achieved full reconciliation of numeric positioning with sequence identity, eliminating distortion from starting points.
  • Universal Application: The transition from a 4-letter DNA alphabet to a 20-letter amino acid alphabet (a 5D hypercube) is mathematically straightforward and requires no algorithmic changes.

Conclusion and Future Work

While USM proves mathematically superior to older CGR methods for encoding, future research will focus on integrating these density maps as direct inputs for non-linear classifiers like Convolutional Neural Networks (CNNs). The authors suggest that the true potential of USM lies in exploring the statistical regularities of the fractal embedded space rather than simple compression or alignment tasks.

Summary updated

Abstract

Motivation: With the advent of Language Models using Transformers, popularized by ChatGPT, there is a renewed interest in exploring encoding procedures that numerically represent symbolic sequences at multiple scales and embedding dimensions. The challenge that encoding addresses is the need for mechanisms that uniquely retain contextual information about the succession of individual symbols, which can then be modeled by nonlinear formulations such as neural networks. Context: Universal Sequence Maps(USM) are iterated functions that bijectively encode symbolic sequences onto embedded numerical spaces. USM is composed of two Chaos Game Representations (CGR), iterated forwardly and backwardly, that can be projected into the frequency domain (FCGR). The corresponding USM coordinates can be used to compute a Chebyshev distance metric as well as k-mer frequencies, without having to recompute the embedded numeric coordinates, and, paradoxically, allowing for non-integers values of k. Results: This report advances the bijective fractal encoding by Universal Sequence Maps (USM) by resolving seeding biases affecting the iterated process. The resolution had two results, the first expected, the second an intriguing outcome: 1) full reconciliation of numeric positioning with sequence identity; and 2) uncovering the nature of USM as an efficient numeric process converging towards a steady state sequence embedding solution. We illustrate these results for genomic sequences because of the convenience of a planar representation defined by an alphabet with only 4 tokens (the 4 nucleotides). Nevertheless, the application to alphabet of arbitrary cardinality was found to be straightforward.
BibTeX
@misc{almeida2025fractal,
  author = {Jonas S. Almeida and Daniel E. Russ and Susana Vinga and Ines Duarte and Lee Mason and Praphulla M. S. Bhawsar and Aaron Ge and Arlindo L. Oliveira and Jeya Balaji Balasubramanian},
  title = {{Fractal Language Modelling by Universal Sequence Maps (USM)}},
  year = {2025},
  eprint = {2508.06641},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.06641},
}
Figure