Exabiome: Advancing Microbial Science through Exascale Computing

Steven A. Hofmeyr, Aydin Buluç, Robert Riley, Rob Egan, Oguz Selvitopi, Leonid Oliker, Katherine A. Yelick, Migun Shakya, Brett Youtsey, Ariful AzadComput. Sci. Eng 2024 · DOI

Introduction

Microbiomes—the complex communities of bacteria, viruses, and fungi inhabiting everything from the human gut to deep-sea vents—are the invisible engines of our planet. Despite their importance, the vast majority (over 99%) of microbes cannot be cultured in a laboratory. To study this "dark matter" of biology, researchers rely on metagenomics: sequencing the total DNA from an environmental sample. However, the sheer volume of data generated by modern sequencing has outpaced traditional computing capabilities. The Exabiome project addresses this by developing exascale-ready tools designed to process petabytes of genomic data, unlocking biological insights that were previously unreachable.

The Metagenomic Challenge

Metagenomic sequencing produces billions of short, fragmented DNA sequences known as "reads." Reconstructing these fragments into complete genomes is a computational problem akin to solving a massive jigsaw puzzle without a reference image. Traditional assembly tools are typically restricted to single, high-memory servers, forcing researchers to analyze samples individually—a process called multiassembly.

This approach has a significant drawback: it fails to detect the "rare biosphere." Organisms present in low concentrations across multiple samples often lack enough data in any single sample to be reconstructed. As datasets grow toward the petabyte scale, the bioinformatics community has hit a "computational wall," necessitating a shift toward distributed-memory exascale systems.

The Exascale Toolkit

The Exabiome project has introduced a suite of high-performance tools designed for the world's most powerful supercomputers, such as Frontier:

  1. MetaHipMer: A distributed-memory metagenome assembler built using the UPC++ programming model. By utilizing distributed hash tables and GPU acceleration, it can scale across thousands of nodes, achieving significant speedups over traditional CPU-based methods.
  2. PASTIS & HipMCL: A pipeline for protein analysis. PASTIS uses distributed sparse matrices to perform massive sequence similarity searches, while HipMCL employs a scalable Markov Cluster (MCL) algorithm to group these sequences into functional families.
  3. kmerprof: A tool for comparative metagenomics that calculates distances between samples based on raw DNA substrings ($k$-mers). This allows for "reference-free" analysis, which is essential for studying environmental microbes that have no known relatives in existing databases.

The Scientific Advantage of Coassembly

The primary breakthrough of the Exabiome project is the ability to perform "coassembly"—assembling all samples in a dataset simultaneously. By pooling data from hundreds or thousands of samples, the software increases the effective "depth" of rare sequences. This allows the assembly of genomes from organisms that would otherwise be indistinguishable from sequencing noise in a traditional multiassembly.

Comparison of read coverage density for sequences found only in coassembly versus those found in both coassembly and multiassemblies.

As shown in the figure above, coassembly captures a distinct population of low-coverage organisms (the peak on the left) that are entirely missed by multiassembly methods. In a study of soil samples from the Great Redox Experiment (GRE), coassembly identified 307 high-quality genomes across 23 phyla, whereas traditional methods found only 38 unique genomes across 9 phyla. Furthermore, coassembly identified over 7,600 viral families, 89% of which were invisible to traditional methods.

Performance and Scalability

The transition to exascale computing has reduced processing times from months to minutes. For instance, MetaHipMer successfully assembled the 98TB Human Microbiome Project (HMP) dataset in just 38 minutes using 9,000 nodes on the Frontier supercomputer.

Strong scaling performance of MetaHipMer showing runtime reduction as node count increases.

While the tools exhibit excellent strong scaling—where adding more nodes significantly reduces runtime—the project also highlights the physical limits of computation. As node counts reach extreme levels, the overhead of communication between nodes begins to compete with the speed of local computation, leading to the diminishing returns observed at the highest scales.

Conclusion

By leveraging exascale computing, the Exabiome project has transformed metagenomics from a data-constrained field into a discovery-driven one. The discovery of over 106,000 novel protein families and thousands of rare viruses demonstrates that we have only scratched the surface of microbial diversity. As these tools continue to evolve, they will become vital for monitoring global ecosystems and predicting future biological threats, providing a clearer window into the microbial universe.

Summary updated

BibTeX
@article{hofmeyr2024exabiome,
  author = {Steven A. Hofmeyr and Aydin Buluç and Robert Riley and Rob Egan and Oguz Selvitopi and Leonid Oliker and Katherine A. Yelick and Migun Shakya and Brett Youtsey and Ariful Azad},
  title = {{Exabiome: Advancing Microbial Science through Exascale Computing}},
  journal = {Comput. Sci. Eng.},
  year = {2024},
  doi = {10.1109/mcse.2024.3402546},
}
Figure