MassComp, a lossless compressor for mass spectrometry data

Ruochen Yang, Xi Chen, Idoia Ochoa

TL;DR

Mass spectrometry (MS) data is exploding in size, yet it's stored uncompressed, leading to massive storage costs. This paper introduces MassComp, a specialized lossless compressor that significantly reduces the size of MS data by up to 59%, making it easier to store and share.

What Problem Does This Paper Address?

Mass spectrometry (MS) is a crucial technique in fields like proteomics and metabolomics, generating vast amounts of data. The MassIVE repository alone holds over 123TB of MS data. A major challenge is that this data is typically stored uncompressed, leading to substantial storage costs and hindering efficient data sharing. While general-purpose compressors exist, they aren't optimized for the specific characteristics of MS data, which primarily consists of (m/z)-intensity pairs. This paper aims to address this by developing a specialized, lossless compression algorithm for MS data.

Methods

MassComp is a lossless compressor designed specifically for the (m/z)-intensity pairs that form the core of MS data. It operates on mzXML files, extracting these numerical pairs. Crucially, MassComp employs different compression strategies for the mass-to-charge ratio (m/z) and intensity values due to their distinct natures.

  • Base64 Decoding: The base64 encoded numerical pairs are first decoded into IEEE 754 standard floating-point formats (single or double precision), automatically detecting the precision.
  • m/z Compression: For m/z values, which tend to be smooth and sequential, MassComp uses a delta encoding approach. It converts values to hexadecimal, calculates differences between adjacent values, and then uses an arithmetic encoder to compress the lengths of leading zeros in these differences.
  • Intensity Compression: Intensity values are more challenging to compress due to their less predictable nature. MassComp uses a "match" compression strategy. It searches for matches of predefined bit patterns within the last 15 compressed intensity values. If a match is found, a pointer to the previous value is stored along with the residual (non-matching) bits. If no match is found, a pointer of zero is used, and the unmatched data is stored.
  • General Compression: The remaining metadata from the mzXML file (everything except the numerical pairs) is compressed using the general-purpose gzip algorithm.

MassComp then reconstructs the original mzXML file from the compressed data.

Datasets

The authors evaluated MassComp using several datasets:

  • MassIVE Repository: Three experiments (MSV000080896, MSV000080905, and MSV000081123) were randomly selected from the MassIVE repository. These experiments contained a total of 600MB, 4GB, and 400MB of mzXML files, respectively.
  • Previously Used Datasets: Files from a previous study ([16]) were also used, specifically raw files 110620_fract_scxB05, 121213_PhosphoMRM_TiO2_discovery, and ADH_100126_mix, which contained double-precision m/z-intensity pairs.

These datasets were converted to mzXML format using MSConvert for analysis.

Key Insights and Ideas

  • Specialization Pays Off: The core insight is that tailoring compression algorithms to the specific statistical properties of MS data (like the smoothness of m/z values) yields significantly better results than general-purpose compressors.
  • Distinct Strategies for m/z and Intensity: Recognizing that m/z and intensity values have different statistical characteristics is key. MassComp's success hinges on applying distinct, optimized compression techniques to each.
  • Lossless is Possible and Beneficial: While some existing specialized compressors are lossy (especially for double-precision data), MassComp demonstrates that significant lossless compression is achievable, preserving data integrity.
  • The "Uncompressed" Status Quo is a Problem: The paper highlights that the common practice of storing MS data uncompressed is a significant bottleneck for storage and dissemination, underscoring the need for solutions like MassComp.

Key Results

MassComp demonstrated impressive compression performance:

  • Numerical Data Compression: On average, MassComp reduced the size of the numerical (m/z)-intensity pairs by 46%, with some cases reaching up to 89%.
  • Comparison to General Compressors:
    • Compared to gzip, MassComp offered an average improvement of 27% in compression ratio.
  • Compared to the state-of-the-art numerical compressor FPC, MassComp showed an average improvement of 40%.
  • Whole File Compression: When applied to entire mzXML files from the MassIVE repository, MassComp achieved an average size reduction of 59%.
  • Double-Precision Data: For double-precision data, MassComp achieved significant compression, reducing file sizes by up to 75.29% (for the ADH file).
  • Comparison with MS-Numpress: While MS-Numpress (a lossy algorithm for double-precision) achieved better compression ratios for double-precision data, MassComp's lossless nature is a critical advantage.

Limitations and Future Work

  • Decompression Speed: The authors acknowledge that MassComp's decompression times are higher than gzip's. This is attributed to the need to reassemble the mzXML file and the use of a multi-symbol arithmetic encoder.
  • Future Work: They suggest improving decompression speed by compressing and decompressing pairs in parallel and exploring binary arithmetic encoders.
  • Lossy Options for Double-Precision: While MassComp is lossless, MS-Numpress offers better compression for double-precision data.
  • Future Work: The authors suggest exploring lossy options for MassComp for double-precision data, emphasizing the need for thorough analysis of how precision loss affects downstream applications.
  • Random Access: The current implementation does not support random access to pairs within different scans.
  • Future Work: Support for random access is suggested as a future enhancement.
  • mzML Support: While MassComp works on mzXML, the authors note that the underlying data structure is similar in mzML, suggesting potential for broader applicability with minor modifications.

Summary updated

Abstract

Background: Mass Spectrometry (MS) is a widely used technique in biology research, and has become key in proteomics and metabolomics analyses. As a result, the amount of MS data has significantly increased in recent years. For example, the MS repository MassIVE contains more than 123TB of data. Somehow surprisingly, these data are stored uncompressed, hence incurring a significant storage cost. Efficient representation of these data is therefore paramount to lessen the burden of storage and facilitate its dissemination. Results We present MassComp , a lossless compressor optimized for the numerical (m/z)-intensity pairs that account for most of the MS data. We tested MassComp on several MS data and show that it delivers on average a 46% reduction on the size of the numerical data, and up to 89%. These results correspond to an average improvement of more than 27% when compared to the general compressor gzip and of 40% when compared to the state-of-the-art numerical compressor FPC . When tested on entire files retrieved from the MassIVE repository, MassComp achieves on average a 59% size reduction. MassComp is written in C++ and freely available at https://github.com/iochoa/MassComp. Conclusions: The compression performance of MassComp demonstrates its potential to significantly reduce the footprint of MS data, and shows the benefits of designing specialized compression algorithms tailored to MS data. MassComp is an addition to the family of omics compression algorithms designed to lessen the storage burden and facilitate the exchange and dissemination of omics data.
BibTeX
@article{yang2019masscomp,
  author = {Ruochen Yang and Xi Chen and Idoia Ochoa},
  title = {{MassComp, a lossless compressor for mass spectrometry data}},
  journal = {BMC Bioinformatics},
  year = {2019},
  doi = {10.1186/s12859-019-2962-7},
}
Figure