TL;DR
Mass spectrometry (MS) data is exploding in size, yet it's stored uncompressed, leading to massive storage costs. This paper introduces MassComp, a specialized lossless compressor that significantly reduces the size of MS data by up to 59%, making it easier to store and share.
What Problem Does This Paper Address?
Mass spectrometry (MS) is a crucial technique in fields like proteomics and metabolomics, generating vast amounts of data. The MassIVE repository alone holds over 123TB of MS data. A major challenge is that this data is typically stored uncompressed, leading to substantial storage costs and hindering efficient data sharing. While general-purpose compressors exist, they aren't optimized for the specific characteristics of MS data, which primarily consists of (m/z)-intensity pairs. This paper aims to address this by developing a specialized, lossless compression algorithm for MS data.
Methods
MassComp is a lossless compressor designed specifically for the (m/z)-intensity pairs that form the core of MS data. It operates on mzXML files, extracting these numerical pairs. Crucially, MassComp employs different compression strategies for the mass-to-charge ratio (m/z) and intensity values due to their distinct natures.
- Base64 Decoding: The base64 encoded numerical pairs are first decoded into IEEE 754 standard floating-point formats (single or double precision), automatically detecting the precision.
- m/z Compression: For m/z values, which tend to be smooth and sequential, MassComp uses a delta encoding approach. It converts values to hexadecimal, calculates differences between adjacent values, and then uses an arithmetic encoder to compress the lengths of leading zeros in these differences.
- Intensity Compression: Intensity values are more challenging to compress due to their less predictable nature. MassComp uses a "match" compression strategy. It searches for matches of predefined bit patterns within the last 15 compressed intensity values. If a match is found, a pointer to the previous value is stored along with the residual (non-matching) bits. If no match is found, a pointer of zero is used, and the unmatched data is stored.
- General Compression: The remaining metadata from the mzXML file (everything except the numerical pairs) is compressed using the general-purpose gzip algorithm.
MassComp then reconstructs the original mzXML file from the compressed data.
Datasets
The authors evaluated MassComp using several datasets:
- MassIVE Repository: Three experiments (MSV000080896, MSV000080905, and MSV000081123) were randomly selected from the MassIVE repository. These experiments contained a total of 600MB, 4GB, and 400MB of mzXML files, respectively.
- Previously Used Datasets: Files from a previous study ([16]) were also used, specifically raw files 110620_fract_scxB05, 121213_PhosphoMRM_TiO2_discovery, and ADH_100126_mix, which contained double-precision m/z-intensity pairs.
These datasets were converted to mzXML format using MSConvert for analysis.
Key Insights and Ideas
- Specialization Pays Off: The core insight is that tailoring compression algorithms to the specific statistical properties of MS data (like the smoothness of m/z values) yields significantly better results than general-purpose compressors.
- Distinct Strategies for m/z and Intensity: Recognizing that m/z and intensity values have different statistical characteristics is key. MassComp's success hinges on applying distinct, optimized compression techniques to each.
- Lossless is Possible and Beneficial: While some existing specialized compressors are lossy (especially for double-precision data), MassComp demonstrates that significant lossless compression is achievable, preserving data integrity.
- The "Uncompressed" Status Quo is a Problem: The paper highlights that the common practice of storing MS data uncompressed is a significant bottleneck for storage and dissemination, underscoring the need for solutions like MassComp.
Key Results
MassComp demonstrated impressive compression performance:
- Numerical Data Compression: On average, MassComp reduced the size of the numerical (m/z)-intensity pairs by 46%, with some cases reaching up to 89%.
- Comparison to General Compressors:
- Compared to gzip, MassComp offered an average improvement of 27% in compression ratio.
- Compared to the state-of-the-art numerical compressor FPC, MassComp showed an average improvement of 40%.
- Whole File Compression: When applied to entire mzXML files from the MassIVE repository, MassComp achieved an average size reduction of 59%.
- Double-Precision Data: For double-precision data, MassComp achieved significant compression, reducing file sizes by up to 75.29% (for the ADH file).
- Comparison with MS-Numpress: While MS-Numpress (a lossy algorithm for double-precision) achieved better compression ratios for double-precision data, MassComp's lossless nature is a critical advantage.
Limitations and Future Work
- Decompression Speed: The authors acknowledge that MassComp's decompression times are higher than gzip's. This is attributed to the need to reassemble the mzXML file and the use of a multi-symbol arithmetic encoder.
- Future Work: They suggest improving decompression speed by compressing and decompressing pairs in parallel and exploring binary arithmetic encoders.
- Lossy Options for Double-Precision: While MassComp is lossless, MS-Numpress offers better compression for double-precision data.
- Future Work: The authors suggest exploring lossy options for MassComp for double-precision data, emphasizing the need for thorough analysis of how precision loss affects downstream applications.
- Random Access: The current implementation does not support random access to pairs within different scans.
- Future Work: Support for random access is suggested as a future enhancement.
- mzML Support: While MassComp works on mzXML, the authors note that the underlying data structure is similar in mzML, suggesting potential for broader applicability with minor modifications.
Summary updated