Bottlenecks and Mass Teleportation in Geometric Tempering

This direction is merged into another direction: local mode entrapment. Its last version stays readable and citable here.

Version 1 of 17 papers15 conceptsfrozen 2026-10-04

Sampling from complex, multi-modal distributions is a fundamental challenge in machine learning. A common "cheat code" to solve this is tempering: instead of sampling from a difficult target distribution $P$ directly, we start with a simple distribution $Q$ (like a Gaussian) and gradually morph it into $P$ through a sequence of intermediate distributions $P_t$.

The standard way to build this bridge is through geometric interpolation, where the log-density is linearly interpolated: $\log p_t = (1-t) \log q + t \log p$. While intuitive, recent research has revealed that this "obvious" path is often the source of extreme computational inefficiency, creating artificial barriers that make sampling exponentially harder than it needs to be.

The Illusion of a Smooth Path

The core appeal of geometric tempering is the assumption that intermediate distributions will inherit the "niceness" of the endpoints. However, Omar Chehab, Anna Korba, Austin J Stromme, and Adrien Vacher have shown that this is a mathematical mirage.

In Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamics, the authors demonstrate that even when both the starting distribution $Q$ and the target $P$ are unimodal and well-conditioned, the intermediate geometric mixtures can become highly multi-modal. Imagine two overlapping bells; the geometric mean doesn't necessarily slide one bell toward the other. Instead, it can create a "double-well" potential with a deep probability valley between them. For a gradient-based sampler like Langevin dynamics, this valley acts as an artificial bottleneck, trapping the sampler in one mode and making the convergence rate worsen exponentially during the transition.

The Problem of Mass Teleportation

When the target distribution is already multi-modal, geometric tempering faces an even steeper challenge. Molei Tao, Wei Guo, and Yongxin Chen identify a phenomenon they call mass teleportation in their analysis of Annealed Importance Sampling (AIS).

In Complexity Analysis of Normalizing Constant Estimation: from Jarzynski Equality to Annealed Importance Sampling and beyond, they argue that geometric paths often require probability mass to vanish in one region of the space and reappear in another without a continuous physical transition. Because geometric interpolation multiplies densities, if the source and target have non-overlapping high-probability regions, the intermediate density in the "middle" remains nearly zero.

This creates a high "path action" or complexity. The sampler cannot "walk" from the source mode to the target mode; it must effectively wait for a rare random jump to discover the new mode. This leads to significant estimation bias and a failure to capture all modes in high-dimensional spaces, as the computational effort required to bridge these modes grows exponentially with their separation.

Beyond Geometric Paths

The limitations of geometric tempering have prompted a shift toward alternative interpolation strategies. If geometric paths (interpolating in "log-space") create bottlenecks, perhaps linear paths (interpolating in "variable-space") are the answer.

Omer Deniz Akyildiz, Paula Cordero-Encinar, and Andrew Duncan explore this in the context of Diffusion Annealed Langevin Monte Carlo (DALMC). In Non-asymptotic Analysis of Diffusion Annealed Langevin Monte Carlo for Generative Modelling, they show that paths derived from diffusion processes—which essentially blur the data into noise—can avoid the artificial multi-modality of geometric tempering.

Similarly, Wei Guo and colleagues propose using reverse diffusion samplers as a superior alternative to geometric AIS. By following the path defined by a stochastic differential equation (SDE) that gradually adds noise, the sampler maintains a more "connected" path between modes. This allows the algorithm to navigate complex, multi-peaked landscapes without the exponential slowdown caused by the "teleportation" requirement of geometric paths.

The takeaway for researchers is clear: the path chosen to connect distributions is just as important as the sampling algorithm itself. While geometric tempering is algebraically simple, its tendency to create artificial barriers makes it a dangerous default for complex distributions.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_bottlenecks_and_mass_teleportation_in_ge_0b3aed46,
  title = {Bottlenecks and Mass Teleportation in Geometric Tempering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/bottlenecks-and-mass-teleportation-in-geometric-tempering/rdr_0b3aed46a40f4761a9a2945689b11160},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_bottlenecks_and_mass_teleportation_in_ge_0b3aed46_v1,
  title = {Bottlenecks and Mass Teleportation in Geometric Tempering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/bottlenecks-and-mass-teleportation-in-geometric-tempering/rdr_0b3aed46a40f4761a9a2945689b11160/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}