In modern machine learning, we often model the world using "hidden" or latent variables. Whether it is the unobserved physical states in a particle physics simulator described by Kyle Cranmer or the underlying "utilities" in ordinal data analyzed by Truyen Tran, these variables allow us to express complex dependencies.
However, these models present a fundamental computational hurdle: Maximum Marginal Likelihood Estimation (MMLE). To find the best model parameters $\theta$, we must maximize the marginal likelihood $p(x|\theta)$, which requires integrating out all possible configurations of the latent variables $z$:
$$p(x|\theta) = \int p(x, z|\theta) dz$$
As David Blei and Carl Doersch note, this integral is almost always intractable in high-dimensional or non-conjugate settings. This "intractability gap" prevents us from using standard gradient descent because we cannot even compute the objective function we want to optimize.
The Failure of Traditional Bounds
The classic response to this intractability is Variational Inference (VI), which optimizes the Evidence Lower Bound (ELBO). While popular, Zhijian Ou and Yunfu Song point out that VI introduces a persistent gap between the bound and the actual likelihood, often leading to biased parameter estimates. Furthermore, in models like Markov Random Fields, Chongxuan Li and Jun Zhu explain that we face a "dual" challenge: the posterior of the hidden states is intractable, and the global partition function is also unknown.
To move beyond these bounds, a growing body of research focuses on stochastic optimization frameworks that treat the latent variables not as something to be "bounded away," but as a dynamical system to be sampled during the optimization process.
Sampling While Optimizing
The core intuition behind this research direction is to replace the impossible integral with a sequence of "cheap" stochastic samples. Instead of waiting for a perfect Expectation (E-step) as in the classic EM algorithm—which Daniel Hsu and Ji Xu show is prone to local optima—we perform a "noisy" update.
Alain Durmus and Marcelo Pereyra formalize this in the SOUL (Stochastic Optimisation via Unadjusted Langevin) framework. The idea is to use the Unadjusted Langevin Algorithm (ULA) to sample from the posterior of the latent variables while simultaneously updating the model parameters.
Why does this work? Iain Souttar and Michela Ottobre frame this as a multiscale dynamical system. If the "sampling" of latent variables happens at a faster timescale than the "optimization" of parameters, the parameter updates effectively "see" the average of the latent states, which converges to the true marginal gradient.
Accelerating Convergence with Particles and Momentum
While simple Langevin sampling works, it can be slow to explore complex posterior landscapes. Recent work has introduced "momentum" into this process. Jen Ning Lim and Adam Michael Johansen developed Momentum Particle Descent (MPD), which blends Nesterov’s acceleration with underdamped Langevin dynamics.
In MPD, you don't just track a single estimate of the latent state; you track a "cloud" of particles. By injecting momentum into both the particles (the $z$ space) and the parameters (the $\theta$ space), the algorithm can bypass small local modes in the posterior and converge to the global maximum of the marginal likelihood much faster than standard stochastic EM.
Achieving Unbiased Estimation
A parallel effort seeks to eliminate the bias inherent in Monte Carlo approximations. Mohammad Norouzi and David Duvenaud introduced SUMO (Stochastically Unbiased Marginalization Objective), which uses a "Russian Roulette" sampling trick to truncate the infinite series required for log-marginal estimation.
Similarly, Kei Ishikawa and Takashi Goda utilize Multilevel Monte Carlo (MLMC) to provide debiased estimates of the evidence and its gradient. These methods are crucial because they allow us to optimize the exact likelihood (in expectation) rather than a lower bound, which is essential for tasks like entropy-regularized reinforcement learning or evaluating generative models where, as Roger B. Grosse and Yuhuai Wu demonstrate, biased metrics can hide significant failures like mode collapse.
Summary of the Mechanism
The shift in this research area is away from static approximations (like fixed variational families) and toward dynamic, simulation-based optimization. By coupling the optimization of $\theta$ with a stochastic process that explores $z$, researchers are able to:
- Bypass Intractability: Use samples to approximate gradients that cannot be computed analytically.
- Reduce Bias: Use momentum, particles, or multilevel sampling to ensure the stochastic gradient is a faithful representation of the true marginal likelihood.
- Scale to High Dimensions: Leverage the efficiency of Langevin-type dynamics, which Valentin De Bortoli shows can handle the high-dimensional latent spaces found in modern deep generative models.