MassComp, a lossless compressor for mass spectrometry data

Ruochen Yang, Xi Chen, Idoia Ochoa

Abstract

Mass spectrometry (MS) is a fundamental tool in proteomics and metabolomics, generating vast quantities of data that challenge existing storage infrastructures. Current storage formats often rely on inefficient text-based encoding, while existing specialized compressors frequently sacrifice data precision. We present MassComp, a domain-specific, lossless compression tool designed to handle the unique numerical properties of MS data. By employing specialized algorithms for mass-to-charge (m/z) and intensity values, MassComp achieves significant size reductions—averaging 59% for entire files—while maintaining 100% data integrity.

Introduction

The rapid advancement of high-throughput mass spectrometry has led to an explosion in data volume. Large-scale repositories like MassIVE now manage over 120 terabytes of data, much of which is stored in XML-based formats such as mzXML or mzML. These formats typically encode numerical data in Base64 strings, a method that is highly inefficient for storage and transmission. While general-purpose compressors like gzip provide some relief, they lack the domain awareness necessary to exploit the specific patterns found in MS data. MassComp addresses this by providing a high-performance, lossless alternative that outperforms both general-purpose and existing specialized numerical compressors.

The Challenge of Mass Spectrometry Data Storage

The primary bottleneck in MS data storage lies in the representation of (m/z)-intensity pairs. In standard mzXML files, these pairs are stored as long strings of Base64-encoded characters within scan elements.

Structure of an mzXML file showing metadata and Base64-encoded numerical data blocks.

As shown in the figure above, a typical file consists of general metadata followed by individual scan information. The numerical data within these scans does not compress well with standard dictionary-based algorithms because the bit-level patterns of floating-point numbers appear random to general-purpose tools. Furthermore, while lossy compressors like MS-Numpress can achieve high compression ratios, they introduce precision errors that may be unacceptable for certain downstream analyses, particularly when dealing with double-precision (64-bit) data.

The MassComp Methodology: Specialized Compression

MassComp utilizes a "divide and conquer" strategy, separating the XML metadata from the numerical peak data. The metadata is compressed using standard gzip, while the (m/z)-intensity pairs are processed through two distinct, specialized algorithms that exploit the physical properties of the data.

m/z Compression

Mass-to-charge (m/z) values within a single scan are typically smooth and monotonically increasing. MassComp leverages this by converting IEEE 754 floating-point values into their hexadecimal representations and applying a variation of delta encoding. By calculating the difference between adjacent values in the hexadecimal domain, the algorithm generates strings with many leading zeros, which are then efficiently compressed using an arithmetic encoder.

Intensity Compression

Intensity values are more stochastic than m/z values, requiring a different approach. MassComp employs a "match" compression algorithm that focuses on the internal structure of the floating-point representation.

The IEEE 754 floating-point standard, illustrating the sign, exponent, and fraction bits targeted by MassComp.

As illustrated in the IEEE 754 standard, floating-point numbers are composed of sign, exponent, and fraction bits. MassComp searches for bit-pattern matches—specifically within the sign and exponent bits—among the 15 most recently compressed values. If a match is found, a pointer is stored; otherwise, the raw data is kept. This strategy is adaptive, with the tool inspecting the first 50 values of each scan to determine the most effective matching parameters for that specific data block.

Performance Evaluation and Results

The effectiveness of MassComp was evaluated using diverse datasets from the MassIVE repository, covering both 32-bit and 64-bit precision in centroid and profile modes. The results demonstrate that MassComp is highly effective at reducing data footprints:

  • Numerical Data Reduction: Achieved an average 46% reduction in size, with peak performance reaching 89% in specific cases.
  • Comparative Performance: MassComp outperformed gzip by an average of 27% and the state-of-the-art numerical compressor FPC by 40%.
  • Whole File Compression: When considering the entire mzXML file (metadata plus numerical data), MassComp achieved an average size reduction of 59%.

Crucially, MassComp maintains bit-for-bit accuracy, making it particularly superior for double-precision datasets where lossy compression is not an option.

Discussion and Future Directions

While MassComp offers superior compression ratios, there are areas for future optimization. Currently, decompression is slower than gzip due to the computational overhead of reassembling the XML structure and the arithmetic decoding process. Additionally, the current single-threaded implementation does not yet take advantage of the highly parallelizable nature of MS scans.

Future work will focus on implementing multi-threaded processing to improve throughput and developing an indexing system to allow researchers to access specific scans without decompressing the entire file. Furthermore, the authors plan to extend native support to the mzML format to broaden the tool's utility within the research community. In conclusion, MassComp demonstrates that by exploiting the underlying physical constraints and bit-level structure of mass spectrometry data, it is possible to achieve high compression ratios without sacrificing scientific precision.

Summary updated

Abstract

Background: Mass Spectrometry (MS) is a widely used technique in biology research, and has become key in proteomics and metabolomics analyses. As a result, the amount of MS data has significantly increased in recent years. For example, the MS repository MassIVE contains more than 123TB of data. Somehow surprisingly, these data are stored uncompressed, hence incurring a significant storage cost. Efficient representation of these data is therefore paramount to lessen the burden of storage and facilitate its dissemination. Results We present MassComp , a lossless compressor optimized for the numerical (m/z)-intensity pairs that account for most of the MS data. We tested MassComp on several MS data and show that it delivers on average a 46% reduction on the size of the numerical data, and up to 89%. These results correspond to an average improvement of more than 27% when compared to the general compressor gzip and of 40% when compared to the state-of-the-art numerical compressor FPC . When tested on entire files retrieved from the MassIVE repository, MassComp achieves on average a 59% size reduction. MassComp is written in C++ and freely available at https://github.com/iochoa/MassComp. Conclusions: The compression performance of MassComp demonstrates its potential to significantly reduce the footprint of MS data, and shows the benefits of designing specialized compression algorithms tailored to MS data. MassComp is an addition to the family of omics compression algorithms designed to lessen the storage burden and facilitate the exchange and dissemination of omics data.
BibTeX
@article{yang2019masscomp,
  author = {Ruochen Yang and Xi Chen and Idoia Ochoa},
  title = {{MassComp, a lossless compressor for mass spectrometry data}},
  journal = {BMC Bioinformatics},
  year = {2019},
  doi = {10.1186/s12859-019-2962-7},
}
Figure