TL;DR
This paper introduces a GPU-accelerated version of the RELION cryo-EM image processing software, dramatically speeding up structure determination. By optimizing computationally intensive steps for GPUs, they've enabled near-atomic resolution analysis on a single workstation in days, democratizing access to high-resolution structural biology.
What Problem Does This Paper Address?
Single-particle cryo-electron microscopy (cryo-EM) is revolutionizing structural biology, allowing researchers to determine the 3D structures of biological molecules at near-atomic resolution. However, the process is computationally intensive, requiring massive amounts of data processing. This computational bottleneck limits the throughput of structure determination and hinders the development of new methods. Specifically, the algorithms used for classifying heterogeneous datasets and refining high-resolution structures demand significant computational resources, often necessitating access to large, expensive high-performance computing clusters. This paper aims to alleviate this bottleneck by leveraging the power of Graphics Processing Units (GPUs) to accelerate these critical steps.
Methods
The core of this work is the re-implementation of computationally demanding parts of the RELION software to utilize GPUs. The primary focus is on the "regularised likelihood optimisation" algorithm, which is central to both classification and refinement.
- GPU Parallelization: The authors expose "low-level parallelism" by treating individual image pixels, orientations, and even entire reference maps as independent tasks that can be processed concurrently on GPUs. This is a significant departure from previous RELION versions that primarily scaled by distributing tasks across multiple CPU cores.
- CUDA Implementation: The GPU acceleration is implemented using NVIDIA's CUDA programming language, leveraging libraries like cuFFT for fast Fourier transforms and CUB/thrust for standard functions.
- Single Precision Arithmetic: To maximize performance and reduce memory requirements on GPUs, the implementation utilizes single-precision floating-point arithmetic for most calculations. They demonstrate that this does not adversely affect the final resolution or quality of the determined structures.
- Hybrid Approach: While most computationally intensive E-step calculations are performed on the GPU using single precision, the M-step (reconstruction) remains on the CPU using double precision to maintain accuracy.
- Optimized Particle Picking: The template-based particle selection algorithm was also accelerated using GPUs, with an additional optimization of applying a low-pass filter to micrographs to reduce computational load without sacrificing picking quality.
Datasets
The paper utilizes several publicly available datasets for evaluation:
- Ribosome Data (EMPIAR entry 10028): Used for 3D classification and refinement benchmarks (Figures 4 & 5).
- B-galactosidase Data (EMPIAR entry 10017): Used for evaluating the performance of the semi-automated particle picking algorithm (Figure 7, Table 1).
- B-galactosidase Workflow (EMPIAR entry 10061): Used to demonstrate a complete high-resolution structure determination workflow on a single workstation (Figure 8).
No new benchmarks were explicitly introduced, but the existing datasets were used to showcase the performance improvements of the GPU-accelerated RELION-2.
Key Insights and Ideas
- Democratizing High-Resolution Cryo-EM: The most significant insight is that powerful GPUs, even on desktop hardware, can now rival or surpass the performance of large CPU clusters for cryo-EM processing. This makes high-resolution structure determination accessible to a much wider range of labs.
- The Power of Low-Level Parallelism: By exposing parallelism at the pixel and orientation level, the authors unlocked substantial speedups that were not possible with previous CPU-centric parallelization strategies.
- Single Precision is Sufficient: The finding that single-precision arithmetic doesn't compromise structural resolution is crucial for GPU acceleration, as it halves memory requirements and significantly boosts performance.
- New Bottlenecks Emerge: As GPU acceleration tackles the primary computational hurdles, the paper highlights that data input/output and other CPU-bound steps can become new bottlenecks, guiding future development.
- Low-Pass Filtering for Picking: The simple yet effective strategy of low-pass filtering micrographs before particle picking significantly reduces computation without impacting accuracy, demonstrating that optimization can come from unexpected places.
Key Results
- Order-of-Magnitude Speedups:
- Image classification and high-resolution refinement were accelerated by more than an order-of-magnitude.
- Template-based particle selection saw accelerations of well over two orders-of-magnitude.
- Workstation Performance: A quad-GPU workstation can outperform a 280-CPU core cluster for 3D classification.
- Reduced Computational Cost: GPU acceleration makes it feasible to run many computationally intensive calculations on workstations or low-cost desktop solutions.
- High-Resolution Achieved: Near-atomic resolution cryo-EM structures can now be calculated in a matter of days on a single workstation.
- Quality Maintained: Single-precision GPU refinement yielded results qualitatively identical to double-precision CPU refinement, as evidenced by Fourier Shell Correlation (FSC) curves.
- Particle Picking Efficiency: Filtering alone provided a ~20-fold performance improvement, and when combined with GPU acceleration, a single GPU achieved ~500x performance gain compared to an 8-core desktop.
Limitations and Future Work
- CPU Bottlenecks: The paper acknowledges that while GPU acceleration has removed major bottlenecks, data input/output and CPU-bound reconstruction steps (M-step) now account for a significant portion of the total execution time. These areas are identified as targets for future optimization.
- Memory Requirements: While memory requirements on GPUs have been reduced, they can still be a limiting factor for very large datasets or high-resolution structures. Future work aims to further reduce GPU memory requirements.
- Generalization of Parallelism: The authors intend to generalize the low-level parallelism to vectorised CPU calculations and potentially open GPU languages like OpenCL to improve performance and coverage across different architectures.
- Further Optimization: The current hybrid approach (GPU for E-step, CPU for M-step) is a compromise. Further optimization of the M-step is a candidate for future work.
Summary updated