Calibrated Forward Folding for Co-eluting Compounds in GC-MS

When two compounds leave a gas chromatography column at nearly the same time, the mass spectrometer records a blend of their fragment patterns. Software then has to pull the blend apart before each piece is matched against a reference library. The most scalable current tool, MSHub, does this separation blindly with non-negative matrix factorization, a method that splits a data matrix into non-negative parts with no knowledge of the instrument. When that separation fails, the result is a "chimeric" spectrum, a mixture that looks like a single compound, and the library search then reports a confident false identification.

Particle and neutron physicists handle a problem with the same structure in a different way. They measure the detector's distortion on known reference sources, write it down as an explicit response model, and then either invert the model with care or push candidate "true" signals forward through it and compare the predictions with raw data. This proposal carries over the second strategy, forward folding. We calibrate each GC-MS run's response from compounds already present in it, predict what each candidate mixture of library compounds would look like on this particular instrument, fit those predictions to the raw counts, and use the quality of the fit to decide whether a proposed identification is real or chimeric.

Core Claim

Blind factorization mixes up three effects: the instrument's own distortion, the chromatographic peak shape, and chemistry. We claim that separating them helps where current tools are weakest, namely low-abundance compounds co-eluting with abundant ones. The separation uses a per-run response calibrated on internal standards, together with a Poisson likelihood that compares library-constrained candidates against raw counts. Concretely, we model the raw ion counts $y_{tm}$ at scan time $t$ and mass-to-charge value $m$ as Poisson with mean

$$\mu_{tm} = g_m \sum_k c_k\, s_k(m)\, h(t - \tau_k;\theta) + b_{tm}.$$

Each term has a specific meaning. The term $g_m$ is the instrument's mass-dependent transmission and detector gain, which is calibrated. The term $s_k(m)$ is candidate $k$'s reference spectrum, taken from NIST or predicted by a neural model. The term $h$ is the elution peak shape, with parameters $\theta$ covering width and tailing; it is also calibrated, and is interpolated across retention time. The remaining quantities, the abundances $c_k$, elution times $\tau_k$, and background $b_{tm}$, are fitted.

The new element is not the peak-fitting idea in itself. What is new is two things borrowed from physics practice. The first is calibrating $g_m$ and $\theta$ from reference compounds, the way High-fidelity unfolding of 252Cf neutron spectra calibrated its detector broadening on Cs-137 and Na-22 Compton edges before unfolding. The second is judging hypotheses in the observed-count space, the way the Daya Bay antineutrino unfolding compared theoretical models after forward folding. Together these produce a per-feature likelihood-ratio statistic that says whether adding a candidate compound is actually supported by the data.

Relation to Prior Work

Two GC-MS works motivate the gap. MSHub automates deconvolution at repository scale using non-negative matrix factorization with Fourier-based alignment. It reports 99.17% correlation to XCMS at about one minute per file. It has no instrument model, however, and it needs a heuristic "Balance Score" to filter out results that do not reproduce across runs. Feature-based Molecular Networking groups features and quantifies them downstream, but it inherits whatever split or chimeric peaks the upstream extractor produces.

Two works supply the methods we reuse. Accelerated time-of-flight mass spectrometry shows that an explicit Poisson detector model with a sparsity-regularized maximum-likelihood inversion can separate overlapping ion signals. That work acts only on the ion-arrival axis and never on chromatographic co-elution. We borrow its likelihood and its evaluation design, in which real single-component scans are overlapped computationally so that ground truth is known. NEIMS predicts 70 eV electron-ionization spectra from molecular structure. It supplies candidate spectra $s_k$ for compounds missing from NIST, and its authors explicitly leave instrument-specific adaptation open. That adaptation is exactly the term $g_m$.

The physics unfolding papers constrain the design. The ALEPH jet analysis (ALEPH archived data) shows that a response model built from mismatched simulation biases the result. We therefore calibrate on in-run standards rather than trusting a generic instrument model. The Daya Bay result argues for comparing in folded space rather than unfolded space, which is why we do not invert the whole chromatogram. Generative unfolding of jets solves a high-dimensional problem with flow models. We deliberately do not follow it: a GC-MS peak window is low-dimensional and has a physical structure, and a learned generator would bring back the uncalibrated model dependence we are trying to remove.

Research Program

We will build a three-part pipeline.

The first part is calibration. Every run contains either an alkane retention index series or deuterated internal standards, or both. From these compounds we fit $g_m$ as a smooth function of $m$, and we fit the peak-shape parameters $\theta$ as smooth functions of retention time, by maximizing the Poisson likelihood. This is the analogue of fitting broadening parameters from Compton edges.

The second part generates candidates. A fast first pass, which can be MSHub itself, proposes feature windows. For each window we retrieve the top NIST candidates, NEIMS-predicted spectra for unlisted structures, and retention-index-compatible compounds.

The third part is folded fitting and testing. Within each window we fit the abundances $c_k$ and elution times $\tau_k$ by maximizing the Poisson likelihood. Sparsity is enforced by greedy forward selection, and we also run an iterative expectation-maximization variant (maximum-likelihood expectation maximization, MLEM, as used for the neutron unfolding) to check that the answer does not depend on the solver. We report a likelihood-ratio statistic for each included compound, calibrated by parametric bootstrap from the fitted model. We also report a goodness-of-fit residual for the window: a large residual flags either a missing compound or a chimera.

The scientific question is whether calibration and the folded-space test lower false annotations under co-elution without costing recall on well-separated peaks.

Baselines

The naive reference point is unmodified vendor or XCMS-style peak picking followed by a direct NIST search, with no deconvolution beyond what the vendor provides. It shows how bad co-elution is without any help.

The strongest existing competitor is MSHub (Algorithmic Learning for Auto-deconvolution). It is the only automated, tuning-free GC-MS deconvolver in the file, and it reports 99.17% correlation to gold-standard manual XCMS processing at about one minute per file. We will run it with its default settings on exactly the same files and search its output spectra against the same library, including the same NEIMS-augmented entries. Our method gets no extra tuning: its calibration uses only standards already in the runs, and any hyperparameter, such as the forward-selection threshold, is fixed on a held-out split. The file contains no reported result for chimera rate or co-elution identification accuracy, so we will measure MSHub's figures ourselves. Beating MSHub means a lower false annotation rate at equal or higher recall on synthetic co-elutions, with no loss on isolated peaks.

For library matching against predicted spectra, the reference point is NEIMS's reported 54.3% recall@1, against 42.6% for CFM-EI, and 91.7% recall@10 on NIST 2017 (NEIMS). That number comes from clean spectra. We will measure how far recall@1 falls under co-elution for each pipeline.

Two ablations isolate the key ingredient. The first, library-seeded fitting, keeps our likelihood and candidates but fixes $g_m = 1$ and uses a generic Gaussian peak with a free width. It removes calibration and nothing else. The second, diagonal correction only, applies the calibrated $g_m$ to MSHub's output spectra before searching. It tests whether calibration helps even without forward folding.

Why not just…?

Why not just use MSHub's factorization and filter low-quality components with its Balance Score? The Balance Score measures how consistent a component is across runs. It says nothing about whether a single spectrum within one run is a blend, and it cannot use library knowledge while separating the signal. That said, MSHub's reported agreement with XCMS is high, and on well-separated peaks we expect to do no better. The synthetic co-elution sweep tests whether MSHub's error is concentrated at small retention-time gaps and high abundance ratios. If it is not, the proposal's premise fails.

Why not just correct the mass-dependent gain and keep everything else unchanged? This is a real threat. Electron-ionization instruments are routinely tuned toward reference ion ratios, so a diagonal $g_m$ may account for much of the benefit for library matching. We include this alternative as the "diagonal correction only" ablation. If it matches the full method on false annotations, we will report the contribution as the calibration step and drop the claim about forward folding.

Why not just seed the factorization with library spectra (semi-blind factorization)? Seeding does bring library knowledge in. Without a calibrated peak shape and a Poisson likelihood, however, it has no principled test for whether an extra component is warranted, so it tends to absorb residual signal into spurious candidates. The library-seeded ablation is essentially this approach. The question it settles is whether calibration and likelihood-based selection, rather than the candidates themselves, produce the gain.

Why not invert the whole chromatogram with Tikhonov regularization (a smoothness penalty on the solution)? Direct inversion of a smoothing response amplifies noise and depends on the regularization strength. In the Daya Bay work, forward folding exposed model discrepancies more sensitively than unfolded comparisons (Daya Bay). We include a Tikhonov deblurring of each window followed by standard search as a secondary baseline.

Proposed Experiments

The primary benchmark copies the ATOF validation design. We acquire, or take from MSHub's benchmark data where available, single-compound GC-MS runs of roughly 200 NIST-listed volatiles and internal standards. We then overlap pairs and triples computationally at controlled retention-time gaps, from 0 to 2 peak widths, and abundance ratios, from 1:1 to 1:100, adding Poisson resampling and real blank-matrix background. Ground truth is exact.

The metrics are four: false annotation rate, recall@1 and recall@10 for each true component, cosine similarity of recovered spectra to clean references, and relative abundance error. For the chimera and missing-compound flag we report AUROC together with the calibration of its false positive rate.

The second experiment tests transfer across instruments. We fit on data from one instrument and test on runs from a second instrument of a different model, measuring how much the calibrated $g_m$ recovers the recall@1 lost to the instrument shift.

The third experiment uses real matrices. We spike known compounds into HS-SPME extracts of the kinds profiled in the cheddar cheese and mango wine studies, and measure identification of the spiked compounds and quantitation linearity against dilution series.

The decisive checks are two. The full method must beat both MSHub and the library-seeded ablation on false annotations in the small-gap, high-ratio region at matched recall. The likelihood-ratio flag must also keep its nominal false positive rate on real matrices. If the diagonal-correction ablation matches the full method, the finding reduces to instrument calibration, and we will report it as such.