HyperSpec: Ultrafast Mass Spectra Clustering in Hyperdimensional Space

Weihong Xu, Jaeyoung Kang, Wout Bittremieux, Niema Moshiri, Tajana RosingJournal of Proteome Research 2023 · DOI

TL;DR

This paper introduces HyperSpec, a novel spectral clustering tool for mass spectrometry data that leverages hyperdimensional computing (HDC) and GPU acceleration. HyperSpec significantly speeds up the processing of massive proteomics datasets, reducing runtime by orders of magnitude while maintaining comparable clustering quality to existing methods.

What Problem Does This Paper Address?

Modern mass spectrometry (MS) experiments generate enormous amounts of data, with current instruments producing gigabytes per hour. Processing this data, particularly for tasks like spectral clustering, has become a major bottleneck. Spectral clustering aims to reduce redundancy by grouping similar MS/MS spectra into consensus spectra, which then speeds up downstream analysis like peptide identification. However, existing spectral clustering tools are often too slow to handle the sheer scale of data generated by today's high-throughput proteomics experiments, with runtimes measured in hours or even days for millions of spectra. This paper addresses the critical need for a spectral clustering method that is both fast and accurate enough to manage these massive datasets.

Methods

HyperSpec employs a multi-stage approach that strategically utilizes both CPU and GPU resources. The core innovation lies in its use of Hyperdimensional Computing (HDC).

  • Spectrum Preprocessing (CPU): This initial stage involves loading, parsing, and cleaning raw spectra. HyperSpec optimizes this by using multiprocessing across multiple CPU cores and a hybrid C++-Python parser for efficient data handling.
  • Bucket Division: Spectra are sorted and divided into "buckets" based on their precursor m/z values. This is a crucial step to reduce the computational complexity, as it limits pairwise comparisons to spectra within the same bucket.
  • HD Encoding (GPU): This is where HDC comes into play. Spectra are transformed into high-dimensional binary vectors called hypervectors (HVs). This encoding captures spectral features (m/z and intensity) in a compact, parallelizable format. The encoding process involves vectorization, quantization, and then a majority vote operation on XORed ID and level hypervectors.
  • HD Distance Computation (GPU): The similarity between spectra is then calculated using a normalized Hamming distance on their HVs. This operation is computationally lightweight, leveraging efficient GPU intrinsics.
  • Clustering (GPU/CPU): Finally, standard clustering algorithms like DBSCAN and hierarchical clustering are applied to each bucket using the computed distance matrices. The paper highlights the use of GPU-accelerated libraries (cuML, fastcluster) for this step.

Key optimizations include bit packing for memory efficiency and batched GPU parallel encoding to reduce data transfer overhead.

Datasets

The authors evaluated HyperSpec on five MS datasets of varying sizes, including data from human cell lines (HEK293, HeLa) and a draft human proteome dataset. These datasets were sourced from the PRIDE database and converted to MGF format. The largest dataset, "Data set-E" (human proteome draft), contained 21.1 million spectra and 131 GB of data, and was used for comprehensive runtime and quality evaluations.

Key Insights and Ideas

  • HDC for Scalability: The core insight is that HDC's binary, high-dimensional representations are inherently parallelizable and well-suited for GPU acceleration, offering a path to overcome the scalability limitations of traditional spectral clustering methods.
  • Preprocessing as a Bottleneck: The paper emphasizes that spectrum preprocessing can account for a significant portion (20-90%) of the total runtime for existing tools, and HyperSpec's optimized CPU-based multiprocessing effectively addresses this.
  • Bucket Division for Efficiency: The strategy of dividing spectra into buckets based on precursor m/z is critical for managing the quadratic complexity of pairwise comparisons, making large-scale clustering feasible.
  • Trade-offs in Clustering Parameters: The study meticulously investigates the impact of HD parameters (dimension D, quantization level Q) and clustering algorithms (DBSCAN vs. hierarchical) on clustering quality, providing guidance for optimal configuration.
  • GPU Acceleration is Key: Offloading HD encoding and distance computation to the GPU is fundamental to HyperSpec's speedup, leveraging the massive parallelism of modern GPUs.

Key Results

  • Massive Speedup: HyperSpec achieved speedups of orders of magnitude compared to existing tools. For a dataset of over 21 million spectra, the clustering runtime was reduced from 4 hours to just 24 minutes.
  • Comparable Clustering Quality: HyperSpec produced results with comparable clustering quality to state-of-the-art spectral clustering tools, as measured by clustered spectra ratio, incorrect clustering ratio, and completeness.
  • Runtime Efficiency: HyperSpec consistently outperformed other fast spectral clustering tools (msCRUSH, falcon, GLEAMS) in terms of total runtime, showing speedups of 10.8× to 15.0× across different datasets.
  • Memory Efficiency: HyperSpec was more memory efficient than some comparable tools like msCRUSH and falcon, with a peak memory consumption of 54 GB for the largest dataset.
  • Scalability: The tool demonstrated excellent scalability with increasing data size and number of spectra.

Limitations and Future Work

  • Parameter Tuning: While HyperSpec offers flexibility, finding the optimal HD parameters (D, Q) and clustering algorithm for specific datasets might still require some experimentation.
  • Further Optimization: The authors suggest several avenues for future work:
  • Optimized Distance Function: Developing a more refined distance function for comparing HVs could further improve clustering quality.
  • Postprocessing Schemes: Implementing postprocessing to split invalid clusters could enhance accuracy.
  • Multi-GPU Support: Parallelizing the clustering step across multiple GPUs could yield near-linear speedups.
  • Near-Storage Processing: Integrating with emerging near-storage processing hardware could offer even greater energy efficiency.
  • Data Compression: The compact HV representation could be explored for MS data compression and direct use in downstream tasks like spectral library searching.

Summary updated

Abstract

As current shotgun proteomics experiments can produce gigabytes of mass spectrometry data per hour, processing these massive data volumes has become progressively more challenging. Spectral clustering is an effective approach to speed up downstream data processing by merging highly similar spectra to minimize data redundancy. However, because state-of-the-art spectral clustering tools fail to achieve optimal runtimes, this simply moves the processing bottleneck. In this work, we present a fast spectral clustering tool, HyperSpec, based on hyperdimensional computing (HDC). HDC shows promising clustering capability while only requiring lightweight binary operations with high parallelism that can be optimized using low-level hardware architectures, making it possible to run HyperSpec on graphics processing units to achieve extremely efficient spectral clustering performance. Additionally, HyperSpec includes optimized data preprocessing modules to reduce the spectrum preprocessing time, which is a critical bottleneck during spectral clustering. Based on experiments using various mass spectrometry data sets, HyperSpec produces results with comparable clustering quality as state-of-the-art spectral clustering tools while achieving speedups by orders of magnitude, shortening the clustering runtime of over 21 million spectra from 4 h to only 24 min.
BibTeX
@article{xu2023hyperspec,
  author = {Weihong Xu and Jaeyoung Kang and Wout Bittremieux and Niema Moshiri and Tajana Rosing},
  title = {{HyperSpec: Ultrafast Mass Spectra Clustering in Hyperdimensional Space}},
  journal = {Journal of Proteome Research},
  year = {2023},
  doi = {10.1021/acs.jproteome.2c00612},
}
Figure