Fundamental limits for weighted empirical approximations of tilted distributions

Sarvesh Ravichandran Iyer, Himadri Mandal, Dhruman Gupta, Rushil Gupta, Agniv Bandhyopadhyay, Achal Bassamboo, Varun Gupta, Sandeep JunejaarXiv 2025 · DOI · arXiv

Abstract

In rare-event simulation, risk management, and climate science, "exponential tilting" is a fundamental technique used to shift probability distributions toward extreme outcomes. While traditionally dependent on known probability density functions, modern applications increasingly rely on data-driven approximations. This paper establishes the theoretical boundaries for tilting distributions when only finite data samples are available. We identify a stark dichotomy: approximating tilted distributions for bounded random variables requires a sample size that grows polynomially with the tilt intensity, whereas unbounded variables (such as Gaussians) necessitate a super-polynomial explosion in sample size to maintain accuracy.

Introduction

Exponential tilting is a cornerstone of importance sampling, allowing researchers to study the "tails" of a distribution—where rare but high-impact events occur—by reweighting the original measure. In contemporary data science, we often lack the underlying density function and must instead rely on a set of raw observations $(X_1, \dots, X_n)$. The standard approach in these scenarios is the Self-Normalized Importance Sampler (SNIS), which assigns weights to existing data points to approximate the desired tilted distribution.

Despite its widespread use, the sample complexity of this approach—specifically how many samples ($n$) are required as the tilt magnitude ($\theta$) increases—has remained poorly understood. This research provides a rigorous characterization of these limits, revealing that the geometric properties of the underlying distribution's support fundamentally dictate the feasibility of data-driven tilting.

Methodology

The study employs a mathematical framework rooted in Empirical Process Theory and Extreme Value Theory to analyze the convergence of the empirical tilted measure to the true tilted distribution.

The Estimator and Metric

The authors evaluate the reweighted empirical estimator $R_{n,\theta}$, which assigns weights proportional to $e^{\theta X_i}$ to each data point. To quantify the accuracy of the approximation, the Kolmogorov-Smirnov (KS) distance is utilized, measuring the maximum discrepancy between the cumulative distribution functions of the true tilted measure and the empirical approximation.

The Complexity Gauge: $M_\theta$

A central contribution of the paper is the identification of $M_\theta$, a quantity related to the squared coefficient of variation of the weights. $M_\theta$ serves as a "difficulty gauge" for the tilting task. The authors demonstrate that the fluctuations and error of the estimator are universally controlled by the ratio $M_\theta / n$.

Scaling and Limits

To understand the behavior of the estimator as $\theta \to \infty$, the authors utilize Poisson Random Measures (PRM). This allows them to characterize the "critical regime" where the sample size is just sufficient to prevent the approximation from collapsing into a degenerate state.

The Boundedness Dichotomy

The most significant finding of the research is the "Boundedness Dichotomy." The relationship between the tilt $\theta$ and the required sample size $n$ undergoes a phase transition based on whether the random variable is bounded:

  1. Bounded Variables: For variables with a finite upper bound (e.g., Beta or Uniform distributions), the sample size $n$ only needs to grow polynomially relative to the tilt $\theta$ to maintain a constant level of accuracy.
  2. Unbounded Variables: For variables with infinite support (even "light-tailed" ones like Gaussian or Exponential distributions), the required sample size $n$ must grow super-polynomially (e.g., exponentially) as $\theta$ increases.

This implies that for many common distributions used in finance and physics, data-driven tilting becomes computationally intractable very quickly as one moves further into the tail.

Regimes of Convergence

The paper identifies three distinct regimes based on the scaling of $M_\theta$ relative to $n$:

  • The Accurate Regime ($M_\theta/n \to 0$): The empirical tilted distribution converges strongly to the true tilted distribution. The error shrinks at the standard rate of $1/\sqrt{n}$, scaled by the difficulty factor $\sqrt{M_\theta}$.
  • The Critical Regime ($M_\theta/n \to c$): The approximation does not converge to a deterministic limit but remains a random measure driven by the extreme values (maxima) of the sample. The approximation is "jittery" and unreliable for precise estimation.
  • The Inaccurate Regime ($M_\theta/n \to \infty$): The estimator suffers from a total collapse. The weights become so concentrated that the entire empirical distribution is effectively represented by a single data point—the sample maximum ($X_{(n)}$).

Conclusion and Future Directions

This work provides the first comprehensive theoretical map of the limits of data-driven tilting. While it confirms that tilting is feasible for bounded data, it issues a cautionary note for researchers working with unbounded distributions: without an astronomical number of samples, empirical tilts of Gaussian-like data will inevitably collapse toward the sample maximum.

Future research may extend these findings to heavy-tailed distributions (the Frechet regime), where the authors suspect tilting is even more constrained. Additionally, while this study focuses on the KS distance, exploring other metrics like the Wasserstein distance could provide further insights into the geometry of tilted empirical measures, particularly in the context of generative AI and diffusion models.

Summary updated

BibTeX
@misc{iyer2025fundamental,
  author = {Sarvesh Ravichandran Iyer and Himadri Mandal and Dhruman Gupta and Rushil Gupta and Agniv Bandhyopadhyay and Achal Bassamboo and Varun Gupta and Sandeep Juneja},
  title = {{Fundamental limits for weighted empirical approximations of tilted distributions}},
  year = {2025},
  eprint = {2512.23979},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.23979},
}
Figure