BERTPhylo: An Efffcient Method to Accelerate Phylogenetic Inference Using a Pretrained DNA Language Model

Wuqin Xu, Danruo Deng, Bian Wu, Hans Peter Comes, Yu Feng, Pan Li, Jinfang Zheng, Guangyong Chen, Pheng‐Ann HengResearch Square 2024 · DOI

Abstract

Understanding the phylogenetic relationships among species is crucial for comprehending major evolutionary transitions, serving as the foundation for many biological studies. Despite the ever-growing volume of sequence data providing a significant opportunity for biological research, constructing reliable phylogenetic trees effectively becomes more challenging for current analytical methods. In this study, we introduce a novel solution to accelerate phylogeny inference using a pretrained DNA language model. Our approach identifies the taxonomic unit of a newly collected sequence using existing taxonomic classification systems and updates the corresponding subtree, akin to surgical corrections on a given phylogenetic tree. Specifically, we leverage a pretrained BERT network to obtain high-dimensional sequence representations, which are used not only to determine the subtree to be updated but also identify potentially valuable regions for subtree construction. We demonstrate the effectiveness of our method, named BERTPhylo, through experiments on our established PlantSeqs dataset, focusing on Embryophyta. Our findings provide the first evidence that phylogenetic trees can be constructed by automatically selecting the most informative regions of sequences, without manual selection of molecular markers. This discovery offers a robust guide for further research into the functional aspects of different regions of DNA sequences, enriching our understanding of biology.

Introduction

Constructing phylogenetic trees—the "trees of life" that map evolutionary relationships—is a cornerstone of biological research. However, the field is currently facing a massive computational bottleneck. As genomic sequencing costs plummet, the volume of data is expanding exponentially. Traditional methods, such as Maximum Likelihood, are "NP-hard," meaning their computational complexity grows super-exponentially with the number of sequences.

While deep learning has been proposed as a solution, existing models often struggle to scale or rely on simulated data that fails to capture the complexity of real-world biology. BERTPhylo addresses this gap by utilizing Large Language Models (LLMs) to make tree-building faster, smarter, and more automated.

Methodology

The core of BERTPhylo is a pretrained BERT network (specifically DNABERT) adapted for genomic data. The workflow consists of three primary stages:

  1. Novelty Detection & Classification: Upon receiving a new sequence, the model employs "Hierarchical Linear Probes" (HLPs) to determine if it belongs to a known group (In-distribution) or a novel one (Out-of-distribution). It classifies the sequence across four taxonomic ranks: class, order, family, and genus.
  2. Subtree Identification: Rather than recalculating the entire global tree, BERTPhylo identifies the specific "subtree" or branch where the new sequence belongs.
  3. High-Attention Region Extraction: Utilizing internal "attention weights," the model identifies the most informative segments of the DNA sequence. By focusing only on these high-value regions for alignment (via MAFFT) and tree inference (via RAxML), the system bypasses the need for full-length sequence alignment, significantly reducing processing time.

Datasets: The PlantSeqs Benchmark

To evaluate BERTPhylo, the authors introduced PlantSeqs, a new benchmark dataset:

  • Composition: 157,742 DNA sequences from 19,887 species (150 genera) within the Embryophyta (land plants) group.
  • Markers: Includes nine commonly used molecular markers from both chloroplast and nuclear genomes (e.g., atpB, matK, rbcL, and ITS).
  • Utility: It provides a diverse, real-world dataset to test taxonomic classification and the detection of previously unseen species.

Key Results and Insights

BERTPhylo demonstrates that AI "attention" correlates with biological information. The model’s self-attention mechanism naturally focuses on regions with high genetic variation, which are most useful for distinguishing species. By treating tree-building as a series of local "surgical corrections," the system avoids the computational traps of global reconstruction.

Performance Comparison

In taxonomic classification, BERTPhylo consistently outperformed other deep learning models like BERTax and DNABERT. When combined with traditional methods like MMseqs2, it achieved state-of-the-art results across various taxonomic ranks, as shown in the performance metrics below.

Table 1: Performance comparison of BERTPhylo against deep learning-based and traditional taxonomic classification methods across multiple taxonomic ranks.

Efficiency and Accuracy

  • Massive Speedup: In a test case involving the order Malpighiales, BERTPhylo updated the tree in just 4 minutes, compared to 20.1 hours for a standard pipeline—a reduction to just 0.33% of the original time.
  • High Accuracy: The model achieved an AUROC of 98.27% for novelty detection at the class level.
  • Topological Integrity: Trees constructed using only "high-attention" regions were significantly more accurate and better resolved than those using "low-attention" regions, confirming that the model captures meaningful evolutionary signals.

Conclusion

BERTPhylo represents a significant step toward automated, high-speed phylogenetics. While performance currently dips at lower taxonomic ranks due to data imbalance, and sequence length is limited by the BERT architecture, the framework's success in land plants suggests broad applicability. Future work will focus on expanding the model to other kingdoms and integrating it with larger foundational genomic models to further enhance accuracy and generalizability.

Summary updated

Abstract

<title>Abstract</title> Understanding the phylogenetic relationships among species is crucial for comprehending major evolutionary transitions, serving as the foundation for many biological studies. Despite the ever-growing volume of sequence data providing a significant opportunity for biological research, constructing reliable phylogenetic trees effectively becomes more challenging for current analytical methods. In this study, we introduce a novel solution to accelerate phylogeny inference using a pretrained DNA language model. Our approach identifies the taxonomic unit of a newly collected sequence using existing taxonomic classification systems and updates the corresponding subtree, akin to surgical corrections on a given phylogenetic tree. Specifically, we leverage a pretrained BERT network to obtain high-dimensional sequence representations, which are used not only to determine the subtree to be updated but also identify potentially valuable regions for subtree construction. We demonstrate the effectiveness of our method, named BERTPhylo, through experiments on our established PlantSeqs dataset, focusing on Embryophyta. Our findings provide the first evidence that phylogenetic trees can be constructed by automatically selecting the most informative regions of sequences, without manual selection of molecular markers. This discovery offers a robust guide for further research into the functional aspects of different regions of DNA sequences, enriching our understanding of biology.
BibTeX
@inproceedings{xu2024bertphylo,
  author = {Wuqin Xu and Danruo Deng and Bian Wu and Hans Peter Comes and Yu Feng and Pan Li and Jinfang Zheng and Guangyong Chen and Pheng‐Ann Heng},
  title = {{BERTPhylo: An Efffcient Method to Accelerate Phylogenetic Inference Using a Pretrained DNA Language Model}},
  booktitle = {Research Square},
  year = {2024},
  doi = {10.21203/rs.3.rs-4611601/v1},
}
Figure