Abstract
In time-sensitive situations, such as when a patient has a critical or rapidly developing condition, it is essential to minimize the time required to process a single genomic sample. In these cases, we recommend using GATK3.8 by splitting the sample into chunks and computing across multiple nodes. This approach achieves a walltime of 3.4 hours at a cost of $41.60 using four c5.18xlarge Amazon Cloud instances. Conversely, for routine analyses or large-scale population studies where cost-effectiveness is paramount, we recommend GATK4. By running multiple samples on a single node, the total walltime for 40 samples is approximately 34.1 hours, resulting in a throughput of 1.18 samples per hour at a cost of only $2.60 per sample.
Introduction
Genomic variant calling—the identification of genetic differences between an individual's DNA and a reference genome—is the industry standard for clinical and research applications. However, as sequencing technology has advanced, the computational processing of this data has become a significant bottleneck. The Genome Analysis Toolkit (GATK) is the gold standard for this process, but its performance is highly dependent on software parameters, threading models, and hardware configurations.
With the transition from GATK3.8 to GATK4, researchers have faced challenges in optimizing their workflows. This paper provides a roadmap for tuning these versions based on two primary objectives: absolute speed for urgent clinical diagnostics and cost-effective throughput for large-scale studies.
Methodology
The study involved a comprehensive scalability analysis conducted on Intel Skylake-based high-performance computing (HPC) nodes. The researchers evaluated several optimization vectors:
- Parameter Tuning: Adjusting internal threading options such as
-nctand-nt. - Java Memory Management: Testing Parallel Garbage Collection (PGC) settings to optimize memory usage in GATK4.
- I/O Optimization: Benchmarking asynchronous read/write options for Samtools and Tribble file formats.
- Parallelization: Implementing data-level parallelization by splitting the genome into chromosomal chunks for multi-node processing.
- Cost Analysis: Translating performance metrics into actual dollar costs using Amazon Web Services (AWS) pricing models.
The benchmarks utilized the NA12878 whole-genome sequencing dataset (~20X depth) mapped to the hg38 human reference genome.
Results and Discussion
The Threading Trap
A critical finding of this study is that increasing thread counts does not linearly improve performance across all GATK tools. In GATK3.8, tools like PrintReads exhibit degraded performance when exceeding three threads due to I/O and memory contention.
As shown in the figure above, while BaseRecalibrator and HaplotypeCaller show initial scaling, the gains diminish significantly at higher thread counts. This "threading trap" necessitates a more nuanced approach to parallelization.
Optimization and Walltime
Systematic parameter tuning resulted in execution time reductions of 29.3% for GATK3.8 and 16.9% for GATK4. However, the study found that for certain tools, walltime actually increases if too many threads are allocated to a single task.
To overcome these local bottlenecks, the authors recommend "across-node" parallelization for urgent cases. By splitting a single sample into chunks and processing them across four physical servers, the total walltime can be reduced to 3.4 hours.
GATK4 for High-Throughput Efficiency
While GATK3.8 excels in single-sample speed when heavily parallelized across nodes, GATK4 is designed for throughput. GATK4 tools are largely single-threaded, which allows users to "pack" multiple samples onto a single multi-core node. This strategy minimizes resource conflict and maximizes hardware utilization, reducing the cost per sample to $2.60—a fraction of the cost of the high-speed GATK3.8 configuration.
Conclusion
Choosing between GATK3.8 and GATK4 depends on the specific requirements of the project. For urgent clinical needs, GATK3.8 with multi-node splitting offers the fastest turnaround. For large-scale research, GATK4 provides superior cost-efficiency. Future work will evaluate the performance of GATK4’s Spark-based tools, which were in beta during this study, to determine if they can bridge the gap between speed and throughput.
Summary updated