A case study for cloud based high throughput analysis of NGS data using the globus genomics system

Krithika Bhuvaneshwar, Dinanath Sulakhe, Robinder Gauba, Álex Rodríguez, Ravi Madduri, Utpal Dave, Łukasz Łaciński, Ian Foster, Yuriy Gusev, Subha MadhavanComputational 2014 · DOI

TL;DR

This paper introduces Globus Genomics, a cloud-based platform built on Galaxy, designed to tackle the massive data challenges of Next-Generation Sequencing (NGS). It offers a user-friendly, scalable, and cost-effective solution for analyzing terabytes of genomic data, simplifying complex workflows for translational research.

What Problem Does This Paper Address?

The explosion of Next-Generation Sequencing (NGS) technologies has led to an unprecedented surge in data volume. Analyzing this data, especially for translational research involving hundreds or thousands of samples, requires significant computational power, robust bioinformatics software, and specialized expertise. Traditional high-performance computing (HPC) solutions, like clusters and grids, are often expensive, complex to manage, and suffer from queueing delays, making them impractical for the rapid, large-scale analyses needed today. Furthermore, many academic and research institutions lack the resources to build and maintain their own extensive local infrastructure. This paper addresses the critical need for a scalable, accessible, and efficient platform to manage and analyze terabyte-scale NGS data, bridging the gap between raw sequencing output and actionable biological insights.

Methods

The core of the solution is Globus Genomics, an enhanced version of the popular Galaxy workflow system. Globus Genomics leverages Amazon Web Services (AWS) cloud infrastructure to provide a Software-as-a-Service (SaaS) offering. Key methodological components include:

  • Enhanced Galaxy Workflow System: It builds upon Galaxy's intuitive drag-and-drop interface for creating complex bioinformatics pipelines, while addressing its limitations in data transfer, storage, and parallel processing for large datasets.
  • Cloud-Native Architecture: The system is built on AWS, utilizing its elastic scaling capabilities to dynamically provision compute resources as needed.
  • Globus Integration: It integrates Globus Transfer for efficient and secure data movement to and from the cloud, and Globus Nexus for identity and group management, enabling Single Sign-On (SSO) and controlled access to projects and data.
  • Computational Profiles: For each analytical tool, the system uses pre-defined "computational profiles" that specify optimal instance types (CPU, memory, storage) to ensure efficient execution and cost-effectiveness.
  • Elastic Scaling and Parallelism: The system supports running multiple workflows in parallel and scales compute resources on demand. It also leverages multi-threaded parallelism for individual tools.
  • Batch Submission: A "batch submit" workflow allows users to process large numbers of samples simultaneously, significantly reducing overall analysis time.
  • HTCondor Scheduler: This component is used for efficient scheduling of pipelines across multiple processors, enabling faster computation.

Datasets

The paper uses publicly available datasets for its case studies:

  • Whole Exome Sequencing (WES) data: 78 samples from a lung cancer study obtained from the European Bioinformatics Institute's Sequencing Read Archive (SRA).
  • Whole Genome Sequencing (WGS) data: A small dataset from a human breast cancer cell line (due to the unavailability of large public WGS fastq files for the Illumina platform at the time of the study).
  • RNA-sequencing (RNA-seq) data: 21 samples from The Cancer Genome Atlas (TCGA) ovarian cancer samples, with raw files downloaded from the CGhub archive.

The paper does not introduce new benchmark datasets but rather demonstrates the system's capabilities on existing, real-world genomic data.

Key Insights and Ideas

  • Cloud as a Scalable Solution for NGS Data Overload: The paper strongly advocates for cloud computing as the most practical and cost-effective solution for handling the ever-increasing volume of NGS data, especially for translational research.
  • Addressing Galaxy's Bottlenecks: Globus Genomics successfully overcomes Galaxy's inherent limitations in data transfer, storage, and single-instance processing for large-scale analyses by integrating cloud resources and Globus services.
  • The Power of "Computational Profiles": Dynamically matching analytical tools to optimized cloud instances based on pre-defined profiles is crucial for achieving both performance and cost efficiency.
  • Globus Integration is Key: The seamless integration of Globus Transfer and Nexus provides essential data management, security, and user access functionalities that are critical for collaborative research.
  • Batch Processing is Essential for Throughput: The "batch submit" feature is a game-changer for translational research, enabling the parallel processing of hundreds of samples, drastically reducing turnaround times.
  • I/O is a Major Bottleneck in Cloud Genomics: The study highlights that while compute can be scaled, I/O performance on shared file systems can become a significant bottleneck, necessitating the use of I/O-optimized instances for certain workflows.

Key Results

  • Significant Time Reduction: The Globus Genomics system enabled the analysis of 78 WES samples in approximately 12 hours (with optimized I/O), a substantial improvement over traditional methods that could take weeks.
  • Scalable RNA-seq Analysis: 21 RNA-seq samples were processed in about 24 hours, generating approximately 2.9 TB of data.
  • Cost-Effectiveness: The use of AWS spot instances and optimized computational profiles led to manageable costs for large-scale analyses, as detailed in Table 1. For example, the WES analysis for 78 samples cost $7.62 (with optimized I/O).
  • User-Friendly Interface: The system provides a web-based interface that is accessible to researchers without extensive programming or cloud computing expertise.
  • Reproducibility and Transparency: The Galaxy framework's inherent tracking of analysis details ensures transparency and reproducibility of results.

Limitations and Future Work

  • I/O Bottlenecks: The paper acknowledges that I/O performance on shared file systems can be a bottleneck, especially when many concurrent applications access the same data. While using provisioned I/O on EBS improves performance, there's a natural limit to concurrent jobs on a shared file system. The team is working on better load balancing and collaborating with AWS for larger scalability.
  • Workflow Scope: The current workflows are primarily designed for Illumina HiSeq platforms. Future work aims to expand support for other platforms and NGS data types.
  • Downstream Analysis: The provided workflows focus on pre-processing and core analysis steps. Downstream analysis steps, which can vary greatly depending on the biological question, are not included but can be added by users through the Galaxy interface.
  • Future Directions: The authors plan to expand the system to support more NGS workflows and platforms. They are also developing features for cataloging dynamic data collections, metadata, and provenance, as well as providing advanced search capabilities for discovering and analyzing datasets.

Summary updated

Abstract

Next generation sequencing (NGS) technologies produce massive amounts of data requiring a powerful computational infrastructure, high quality bioinformatics software, and skilled personnel to operate the tools. We present a case study of a practical solution to this data management and analysis challenge that simplifies terabyte scale data handling and provides advanced tools for NGS data analysis. These capabilities are implemented using the "Globus Genomics" system, which is an enhanced Galaxy workflow system made available as a service that offers users the capability to process and transfer data easily, reliably and quickly to address end-to-endNGS analysis requirements. The Globus Genomics system is built on Amazon 's cloud computing infrastructure. The system takes advantage of elastic scaling of compute resources to run multiple workflows in parallel and it also helps meet the scale-out analysis needs of modern translational genomics research.
BibTeX
@inproceedings{bhuvaneshwar2014case,
  author = {Krithika Bhuvaneshwar and Dinanath Sulakhe and Robinder Gauba and Álex Rodríguez and Ravi Madduri and Utpal Dave and Łukasz Łaciński and Ian Foster and Yuriy Gusev and Subha Madhavan},
  title = {{A case study for cloud based high throughput analysis of NGS data using the globus genomics system}},
  booktitle = {Computational},
  year = {2014},
  doi = {10.1016/j.csbj.2014.11.001},
}
Figure