Extreme data compression for Bayesian model comparison

Alan Heavens, Arrykrishna Mootoovaloo, Roberto Trotta, Elena Sellentin

TL;DR

Researchers have demonstrated that massive astronomical datasets can be compressed down to just a few numbers—one per model parameter—without losing any accuracy in Bayesian model comparison. By using the MOPED (Massive Optimal Data compression) algorithm, they show that deciding between competing scientific theories (like whether the universe is flat or curved) can be done just as precisely with three numbers as it can with thousands of data points, significantly slashing the computational cost.

What Problem Does This Paper Address?

In modern cosmology, datasets are exploding in size. Missions like Euclid or the Planck satellite generate millions to billions of data points. To turn this data into science, researchers use Bayesian model comparison, which calculates the "Evidence" (or marginal likelihood) to decide which theory best explains the observations.

The problem is that calculating Evidence is computationally "expensive." It requires integrating over complex, high-dimensional parameter spaces. While a technique called MOPED was already known to be "lossless" for parameter estimation (finding the best-fit values for a single model), it wasn't clear if it would work for model comparison (choosing between two different models). If information is lost during compression, the resulting "Bayes factor" (the ratio of evidences) might lead to the wrong scientific conclusion.

Methods

The authors utilize and extend the MOPED algorithm.

  • How it works: MOPED uses the "score function"—the gradient of the likelihood with respect to the model parameters—to create weight vectors. These vectors compress a large data vector (length $n$) into a tiny summary vector (length $p$, where $p$ is the number of parameters).
  • The Innovation: The authors show that for nested models (where one model is a simplified version of another), you only need the compressed coefficients from the more complex model. For non-nested models, they propose a "union" approach: concatenating the MOPED coefficients from both models into a single, slightly larger compressed dataset.
  • Score Compression: They also generalize the approach to non-Gaussian likelihoods using "score compression," showing that the same principles of information preservation apply.

Datasets

The paper validates the method using two primary approaches:

  1. Toy Nonlinear Models: A synthetic setup comparing a sine-wave model against a quadratic model to test how the compression handles non-linearities and non-nested scenarios.
  2. Pantheon+SH0ES Supernova Dataset: A real-world cosmological dataset consisting of 1,701 supernovae. The authors focused on a subset of 1,590 supernovae ($z > 0.01$) to compare a "flat universe" model (2 parameters) against a "curved universe" model (3 parameters).

Key Insights and Ideas

  • Lossless by Design: For linear models with Gaussian noise, the Bayes factor calculated from compressed data is mathematically identical to the one from the full dataset.
  • The "Cancellation" Miracle: Bayesian Evidence is often criticized for having high "sampling variance" (it changes a lot if you just get a new batch of noisy data). However, the authors show that this noise is mostly common to both models. When you take the ratio (the Bayes Factor), the noise cancels out, making it a stable and reliable statistic even under extreme compression.
  • Stability: Interestingly, extreme compression actually reduces the sampling variance of the Evidence itself, making the individual Evidence values more stable than those derived from the full, noisy dataset.
  • Efficiency: Because the compressed data is so small, the "covariance matrix" used in the likelihood calculations shrinks from a massive $1590 \times 1590$ matrix to a tiny $3 \times 3$ matrix, making every step of the analysis much faster.

Key Results

  • Extreme Compression Ratio: In the supernova example, the authors compressed 1,590 data points into just 3 numbers.
  • High Fidelity:
    • Full Dataset Log Bayes Factor: $-0.92 \pm 0.05$
    • MOPED Compressed Log Bayes Factor: $-0.94 \pm 0.05$
    • The results are virtually identical, with the tiny difference attributed to the fact that the supernova model is slightly non-linear.
  • Parameter Agreement: The marginal posteriors (the "blobs" on the plots showing the most likely parameter values) for the compressed data perfectly overlapped with the full dataset results.

Limitations and Future Work

  • Taylor Expansion Dependency: The MOPED algorithm relies on a linear Taylor expansion around a "fiducial" (reference) point. If the model is extremely non-linear or the initial guess for the parameters is very far from the truth, the compression might lose some information.
  • Gaussian Assumptions: Much of the formal proof for "lossless" compression assumes Gaussian noise. While the authors show it works well for the supernova data, more complex "non-Gaussian" data (like galaxy shapes in weak lensing) might require more sophisticated score compression.
  • Future Directions: The authors suggest this technique will be vital for Simulation-Based Inference (SBI), where we don't have a simple formula for the likelihood and must rely on running thousands of expensive simulations. Compression makes these simulations much more manageable.

Summary updated

Abstract

We develop extreme data compression for use in Bayesian model comparison via the MOPED algorithm, as well as more general score compression. We find that Bayes factors from data compressed with the MOPED algorithm are identical to those from their uncompressed datasets when the models are linear and the errors Gaussian. In other nonlinear cases, whether nested or not, we find negligible differences in the Bayes factors, and show this explicitly for the Pantheon-SH0ES supernova dataset. We also investigate the sampling properties of the Bayesian Evidence as a frequentist statistic, and find that extreme data compression reduces the sampling variance of the Evidence, but has no impact on the sampling distribution of Bayes factors. Since model comparison can be a very computationally-intensive task, MOPED extreme data compression may present significant advantages in computational time.
BibTeX
@article{heavens2023extreme,
  author = {Alan Heavens and Arrykrishna Mootoovaloo and Roberto Trotta and Elena Sellentin},
  title = {{Extreme data compression for Bayesian model comparison}},
  journal = {Journal of Cosmology and Astroparticle Physics},
  year = {2023},
  doi = {10.1088/1475-7516/2023/11/048},
}

Loading related content...

Figure