High-Dimensional Sampling Beyond Log-Concavity

383 concepts280 papers1 proposal

In the worlds of optimization and machine learning, we value convexity because it guarantees that any local minimum is a global minimum. When we shift from optimization to probabilistic sampling, the mathematical equivalent of convexity is log-concavity.

A probability distribution $p(x) \propto e^{-U(x)}$ is log-concave if its negative log-density—the energy landscape $U(x)$—is a convex function. When a distribution is log-concave, it is beautifully well-behaved: it has a single peak (unimodal), and its probability mass fades predictably as you move away from the center. As demonstrated by Arnak S. Dalalyan in Further and stronger analogy between sampling and optimization: Langevin Monte Carlo and gradient descent, sampling from a log-concave distribution behaves like a continuous extension of gradient descent. This allows algorithms like Langevin Monte Carlo (LMC) to navigate the space efficiently, yielding clean, non-asymptotic convergence bounds.

However, real-world data rarely fits into this neat box. Natural images, deep neural network posteriors, and complex physical systems are fundamentally multimodal and non-convex. This creates a significant gap between theory and practice.

The Theoretical Wall of Non-Convex Landscapes

When sampling algorithms encounter non-log-concave distributions, traditional mathematical guarantees collapse. For instance, Gefei Wang et al. in Deep Generative Learning via Schrödinger Bridge and Dohyun Kwon et al. in Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance note that while diffusion and score-based generative models achieve stunning empirical success on complex image manifolds, their core convergence proofs frequently assume log-concavity.

Without log-concavity, an algorithm can easily become trapped in local modes, resulting in exponential mixing times. Traditional proofs rely on global functional inequalities, such as the Log-Sobolev Inequality (LSI) or Poincaré inequalities, which dictate how fast a diffusing particle system reaches equilibrium. As explained by Holden Lee et al. in Convergence of score-based generative modeling for general data distributions, these global inequalities are routinely violated by real-world data concentrated on low-dimensional manifolds or split across multiple distinct modes.

To bridge this gap, a major research direction focuses on breaking past the log-concavity constraint using three main strategies: relaxing functional inequalities, structurally decomposing the target space, and leveraging alternative geometries.

Relaxing Constraints via Weaker Functional Inequalities

Instead of demanding strict log-concavity, can we find weaker geometric conditions that still guarantee efficient sampling?

Bin Yang et al. achieved a significant theoretical breakthrough in Non-asymptotic Error Bounds in W2-Distance with Sqrt(d) Dimension Dependence and First Order Convergence for Langevin Monte Carlo beyond Log-Concavity. They proved that Langevin Monte Carlo can maintain a sharp dimension dependence of $O(\sqrt{d})$ even on non-convex landscapes, provided the target distribution satisfies a Log-Sobolev Inequality rather than strict convexity.

Strict Log-Concavity (Convex Energy) 
   │  ▼ (Historically required for fast O(√d) bounds)
Log-Sobolev Inequality (LSI) 
   │  ▼ (Allows non-convexity, preserves O(√d) bounds)
Poincaré / Talagrand Inequalities 
      (Weaker constraints, handles highly multimodal spaces)

Similarly, Sinho Chewi et al. in Analysis of Langevin Monte Carlo from Poincare to Log-Sobolev bypassed log-concavity entirely to establish convergence guarantees in Rényi divergence under broader isoperimetric conditions. For particle-based variational methods like Stein Variational Gradient Descent (SVGD), which typically require strict regularity, Adil Salim et al. leveraged Talagrand’s Inequality T1 in A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1 to deliver the first dimension-dependent complexity bounds for non-log-concave target distributions.

Slicing and Tilting Complex Distributions

If a global landscape is too rugged to sample directly, another approach is to mathematically alter or decompose the distribution into well-behaved components.

Florentin Guth et al. introduced this concept through Conditionally Strongly Log-Concave Generative Models. Instead of modeling a complex, multiscale distribution all at once, the authors use wavelet packet projectors to factorize the distribution into a sequence of conditional "slices." Because each slice is mathematically forced to be strongly log-concave, the framework enables efficient learning and sampling with provable theoretical guarantees, bypassing the typical "black box" issues of deep generative architectures.

Alternatively, rather than slicing the space, one can smooth out the traps. In Sampling through Algorithmic Diffusion in non-convex Perceptron problems, Elizaveta Demyanenko et al. demonstrated that adding a specific logarithmic potential—a process known as "tilting"—to a highly non-convex landscape can eliminate problematic local minima. This modification makes previously intractable discrete and non-convex spaces efficiently samplable via diffusion algorithms.

Leveraging Alternative Geometries in Variational Inference

Variational Inference (VI) traditionally struggles when moving outside the comfort zone of log-concavity. For example, Edward Challis et al. showed in Concave Gaussian Variational Approximations for Inference in Large-Scale Bayesian Linear Models that Gaussian variational bounds lose their guaranteed concavity when the underlying likelihood components are non-log-concave, severely complicating global optimization.

To resolve this, researchers are shifting their perspective from the standard Euclidean space of parameters to the geometry of probability distributions themselves. Michael Ziyang Diao et al. developed a framework in Forward-Backward Gaussian Variational Inference via JKO in the Bures-Wasserstein Space that treats variational optimization as a gradient flow within the Bures-Wasserstein space. By leveraging this natural optimal transport geometry, the authors provided the first theoretical guarantees for finding stationary points in non-log-concave settings.

By rethinking the mathematical foundations of how we bound, slice, and navigate probability landscapes, this research direction is systematically dismantling the log-concavity barrier—bringing rigorous theoretical guarantees to the complex, multimodal distributions encountered in modern deep learning.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 298 papers
  2. Version 1 · 2026-10-04 · 350 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_log_concavity_assumption_75c57a38,
  title = {log-concavity assumption},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/log-concavity-assumption/rdr_75c57a3847184f08bab3b16abb84cb9b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_log_concavity_assumption_75c57a38_v2,
  title = {log-concavity assumption},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/log-concavity-assumption/rdr_75c57a3847184f08bab3b16abb84cb9b/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake