Bayesian Multivariate Density-Density Regression

Khai Nguyen, Yang Ni, Peter MuellerarXiv 2025 · DOI · arXiv

TL;DR

Researchers have developed the first Bayesian framework for Multivariate Density-Density Regression (DDR), allowing scientists to predict an entire multi-dimensional distribution from another. By combining a generalized Bayesian approach with the Sliced Wasserstein distance, the model efficiently captures complex biological relationships in single-cell data that traditional "mean-based" methods completely miss.

What Problem Does This Paper Address?

In standard regression, we predict a number (scalar) or a list of numbers (vector). But what if your data points are actually clouds of points—specifically, probability distributions?

This is a major challenge in fields like single-cell genomics. When studying how different cell types communicate, researchers often look at "ligands" (signals sent) and "receptors" (signals received). Traditional methods use "pseudo-bulk" analysis, which averages the gene expression across all cells in a sample. This is like trying to understand a city's economy by only looking at the average income; you lose all the information about inequality, diverse industries, and sub-populations.

Existing "Density-Density Regression" (DDR) methods were largely restricted to one-dimensional data because they relied on quantile functions, which don't have a universally accepted definition in higher dimensions. This paper tackles the multivariate problem: regressing a high-dimensional distribution onto another, even when they reside in spaces with different dimensions.

Methods

The authors introduce a Generalized Bayesian framework. Instead of trying to specify a complex, likely-incorrect likelihood function for high-dimensional distributions, they use a loss-based approach.

  1. Sliced Wasserstein (SW) Distance: This is the "secret sauce." The Wasserstein distance (or "Earth Mover's Distance") is a powerful way to compare distributions, but it's computationally expensive in high dimensions. SW solves this by projecting the high-dimensional distributions onto many random 1D lines, computing the easy 1D Wasserstein distance, and averaging the results. It is fast, scalable, and avoids the "curse of dimensionality."
  2. The Mapping ($f_\phi$): They model the relationship using a function that "pushes forward" the predictor distribution to match the response distribution. While they primarily use linear and quadratic mappings for parsimony, the framework is flexible.
  3. MALA Sampler: To perform Bayesian inference, they use the Metropolis-adjusted Langevin algorithm (MALA). They leverage the fact that the SW distance is differentiable (using Danskin’s envelope theorem) to calculate gradients, allowing for efficient exploration of the parameter space.
  4. Horseshoe Priors: To handle the high number of potential gene interactions, they use "horseshoe" shrinkage priors, which help the model automatically identify the most important predictors (sparsity).

Datasets

  • Simulated Data: The authors created synthetic bivariate Gaussian and mixture-of-Gaussian datasets to prove that their model could recover true parameters and that the posterior distribution concentrates as the sample size increases (posterior consistency).
  • OneK1K Dataset: A massive, real-world population-scale single-cell RNA-seq dataset containing gene expression from 439 donors. They focused on four cell types: B cells, T cells, NK cells, and monocytes, specifically looking at ligand and receptor gene sets.

Key Insights and Ideas

  • Beyond the Mean: The most striking "aha moment" is seeing the model handle multimodal distributions. If a response distribution has two distinct clusters, a traditional multivariate linear regression (MLR) will predict a single point in the middle (where no data actually exists). Bayesian DDR, however, predicts the entire multi-peaked shape.
  • Dimension Agnostic: The model doesn't care if your input distribution is 6-dimensional and your output is 15-dimensional. This is crucial for biology, where the number of signaling ligands rarely matches the number of receptors.
  • Graph Discovery: By treating the Relative Predictive Error (RPE) as a measure of "connection strength," the authors can infer a directed communication network between cell types. They found, for example, that B-cell ligands are much better at predicting T-cell receptors than the other way around.
  • Theoretical Rigor: The paper doesn't just provide a tool; it provides a proof of posterior consistency, ensuring that as you collect more samples, the model's "beliefs" will eventually converge on the true underlying relationship.

Key Results

  • Superior Fit: In simulations involving mixtures of Gaussians, Bayesian DDR provided robust fits to high-probability regions where traditional MLR failed.
  • Predictive Accuracy: On the OneK1K dataset, the model achieved a mean Relative Predictive Error (RPE) of ~0.23 for T-cell receptors predicted by B-cell ligands, indicating a strong, learnable relationship.
  • Biological Validation: The model identified specific significant interactions, such as IFN-$\gamma$ being a critical predictor for the IGHA receptor, and CD40L being essential for predicting a suite of receptors including CD40 and BAFFR.

Limitations and Future Work

  • Parametric Limits: The current implementation uses relatively simple linear or quadratic mappings. While effective, these might not capture extremely non-linear biological "logic gates."
  • Computational Mixing: The Sliced Wasserstein loss is non-convex, which can sometimes lead to poor mixing in the MCMC chains (the sampler getting "stuck").
  • Non-Euclidean Data: The current SW distance assumes data lives in Euclidean space. Future versions might need to adapt to distributions living on manifolds (like the hypersphere).
  • Non-parametrics: The authors suggest moving toward Bayesian nonparametric DDR to allow the mapping function itself to be even more flexible.

Summary updated

Abstract

We introduce a novel and scalable Bayesian framework for multivariate-density-density regression (DDR), designed to model relationships between multivariate distributions. Our approach addresses the critical issue of distributions residing in spaces of differing dimensions. We utilize a generalized Bayes framework, circumventing the need for a fully specified likelihood by employing the sliced Wasserstein distance to measure the discrepancy between fitted and observed distributions. This choice not only handles high-dimensional data and varying sample sizes efficiently but also facilitates a Metropolis-adjusted Langevin algorithm (MALA) for posterior inference. Furthermore, we establish the posterior consistency of our generalized Bayesian approach, ensuring that the posterior distribution concentrates around the true parameters as the sample size increases. Through simulations and application to a population-scale single-cell dataset, we show that Bayesian DDR provides robust fits, superior predictive performance compared to traditional methods, and valuable insights into complex biological interactions.
BibTeX
@misc{nguyen2025bayesian,
  author = {Khai Nguyen and Yang Ni and Peter Mueller},
  title = {{Bayesian Multivariate Density-Density Regression}},
  year = {2025},
  eprint = {2504.12617},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.12617},
  howpublished = {arXiv},
}
Figure