Many systems hand us moments instead of samples. Program analyzers for probabilistic loops compute exact means, variances and higher moments symbolically. Kinetic gas solvers evolve moment vectors. Robust classifiers summarize training data by its empirical moments. Each then asks a yes-or-no question: is this a signal or background, hypothesis A or B, is the tail probability above a threshold?
Classical mathematics says that for heavy-tailed distributions, moments cannot fully answer such questions. Two different distributions can share every moment, a property called moment indeterminacy. This proposal turns that qualitative fact into a measurable sensitivity limit. We define the indeterminacy penalty as the extra error, beyond the best possible error, that any test using only integer moments must incur. We show how to compute it with a linear program. We then test which cheap substitute statistics remove it.
The two-sentence version is as follows. For heavy-tailed data, collecting more integer moments stops helping at some point. We compute exactly where, and how large the leftover error is. Then we show that a small set of fractional or logarithmic moments, chosen using one observation from the physics literature, closes most of that gap at a cost comparable to the integer moments they replace.
Core Claim
Fix two hypotheses. Each one specifies the first $K$ moments of the data distribution only up to a tolerance $\delta$. Let $\mathcal{P}_j(K,\delta)$ be the set of all distributions consistent with hypothesis $j$. The best worst-case error of any test against these convex sets follows from Le Cam's two-point argument:
$$ E^\star(K,\delta) \;=\; \tfrac12\Big(1 - \inf_{P_0\in\mathcal{P}_0(K,\delta),\,P_1\in\mathcal{P}_1(K,\delta)} \mathrm{TV}(P_0,P_1)\Big), $$
Here TV is the total variation distance, the largest difference in probability the two distributions can assign to any event. Consider what happens as $K\to\infty$ and $\delta\to 0$.
When the true distributions are determinate (uniquely fixed by their moments), each set shrinks to a single distribution. $E^\star$ then falls to the Bayes error $\tfrac12(1-\mathrm{TV}(P_0^{\text{true}},P_1^{\text{true}}))$, the lowest error any test could achieve with full knowledge of both distributions.
When the true distributions are indeterminate, each set shrinks only to the family of all distributions sharing that moment sequence, and this family is infinite. The limit of $E^\star$ therefore sits strictly above the Bayes error. We call the difference the indeterminacy penalty.
We claim three things. The penalty is computable to controlled accuracy. It is large in practically relevant families. It is mostly removed by moment constraints indexed by a continuous range of exponents rather than by the integers.
Our reason for expecting the third point comes from Moment Problem in Replica Method. That paper shows that integer moments of a partition function fail to pin down $\mathbb{E}[\log Z]$ because the integers have no accumulation point, so many analytic functions interpolate them. A family of fractional absolute moments $\mathbb{E}|X|^s$ with $s$ ranging over an interval does have accumulation points, so that obstruction disappears. We will test this empirically with a handful of exponents.
Relation to Prior Work
The paper that supplies our method is Robust Multi-Hypothesis Testing with Moment-Constrained Uncertainty Sets. It builds minimax tests over moment-constrained sets. It solves them exactly on finite alphabets by a linear program obtained through Lagrangian duality, and approximates them on continuous spaces by an $\epsilon$-net with error proportional to $\epsilon$. It also shows that these sets violate the classical condition (joint stochastic boundedness) needed for a single least-favorable distribution. We reuse its linear program. That paper never asks what happens when the moment sets fail to shrink. We make exactly that question the object of study.
Two papers motivate the regime. Analytical properties of generalized Gaussian distributions proves that the generalized Gaussian family with shape $p$, with density proportional to $\exp(-|x|^p)$, is determinate for $p\ge1$ and indeterminate for $p<1$. This gives us a single dial that crosses the threshold. Constructive Approximation under Carleman's Condition proves quantitative polynomial-approximation and learning rates only when Carleman's condition holds. Carleman's condition is a classical growth test on the moments that guarantees determinacy. Our experiments probe the region that paper excludes: how error behaves as $p\to1^-$ and beyond.
Several papers constrain the design. The moment-reconstruction methods in Distribution Estimation for Probabilistic Loops and Moment-based Density Elicitation with Applications in Probabilistic Loops are maximum entropy, Gram-Charlier series, and K-series, which expands the density in orthogonal polynomials around a reference distribution. All three return a single density from finitely many moments. Inference of Probabilistic Programs with Moment-Matching Gaussian Mixtures explicitly assumes determinacy. Each of these silently picks one member of the indeterminacy family. We measure how wrong a test built on that one member can be compared with the worst member.
Research Program
The first component is the penalty calculator. We discretize the real line on a grid that is adaptive and log-spaced in the tails, and solve the linear program from the robust-testing paper to obtain $E^\star(K,\delta)$. The grid must carry enough tail mass to represent heavy-tailed members of the moment family, so truncation is the main numerical risk. We control it by enlarging the grid until $E^\star$ stabilizes. As an independent check, we use known explicit indeterminate pairs from the Stieltjes construction for the log-normal: densities $f(x)(1+\epsilon\sin(2\pi\log x))$ that share all moments with $f$. These pairs give analytic lower bounds that the solver must reproduce.
The second component replaces integer constraints with alternative statistic families under an equal budget of $K$ constraints:
- fractional absolute moments $\mathbb{E}|X|^{s}$ at exponents $s$ spread over $(0,2)$, together with the probability that $X>0$;
- integer moments of $\log|X|$;
- values of the characteristic function at $K$ frequencies.
For each family we trace $E^\star$ as a function of $K$ and record whether it reaches the Bayes error.
The third component introduces finite samples. With $n$ samples, the tolerance $\delta$ is no longer fixed. It must cover the sampling noise of each empirical statistic, and for high integer moments of heavy-tailed data that noise is enormous. High-order integer moments may have huge or even infinite variance. This is the sample-scarcity limit from counting experiments, where uncertainty shrinks like $1/\sqrt{n}$ only if variances are finite. We set per-statistic tolerances by bootstrap. We then report the sample size needed to reach a target error, as a function of $p$ and of the statistic family.
Baselines
The naive reference point is the nearest-moment rule: assign the data to the hypothesis whose first $K$ integer moments are closest. Robust Multi-Hypothesis Testing with Moment-Constrained Uncertainty Sets proves this rule is exponentially consistent as $n$ grows, and that paper's minimax test beats it. We run it on identical data and $K$.
The strongest existing method is that paper's minimax integer-moment test, run with our solver, the same grid, and the same bootstrap tolerances. The file reports no numerical accuracies for it on Actitracker or on synthetic Gaussians. No reported result was found on Lacuna for this test's error rates, so we reproduce it ourselves. Beating it means a lower worst-case error and a lower empirical error at equal $K$ and $n$, in the indeterminate regime $p<1$. Separately, the two tests must agree to within solver tolerance at $p\ge1$, where we predict no penalty.
A reconstruction-based baseline fits a maximum-entropy density and a K-series density from the same moments, following Distribution Estimation for Probabilistic Loops and Moment-based Density Elicitation with Applications in Probabilistic Loops. It then runs a plug-in likelihood-ratio test on the reconstructed densities, under the same moment budget.
The sample-based upper reference is a likelihood-ratio test with the true densities. It attains the Bayes error and is not a fair competitor, but it marks the floor.
The key ablation holds the solver, $K$, and $n$ fixed and changes only the statistic family. This separates the effect of the exponent set from everything else.
Why not just…?
Why not just use more integer moments? In the indeterminate regime, the theory predicts that $E^\star$ plateaus above the Bayes error regardless of $K$. With samples, high moments also become noisier, so adding them should eventually hurt. The $K$-sweep in Experiment 1 directly shows whether the plateau exists. If $E^\star$ instead keeps falling to the Bayes error, the proposal's premise fails.
Why not just take logs and use ordinary moments? This is a real threat. Fractional moments of $|X|$ are exactly the moment-generating function of $\log|X|$, and for both the log-normal and generalized Gaussians with $p<1$, the variable $\log|X|$ is light-tailed and determinate. We therefore treat log-moments as a first-class candidate rather than a straw man. Discarding the sign is handled by also recording the probability that $X>0$. If log-moments match fractional moments everywhere, the contribution reduces to two parts: the penalty measurement, and the practical rule "test on log-moments when tails are heavy." That rule is still useful, because the current moment-based tools do not follow it.
Why not just test on raw samples with a nonparametric two-sample test or Monte Carlo? When samples exist, a sample-based test such as Kolmogorov–Smirnov should approach the Bayes error, and we include one as a reference. The regime here, however, is moment-only access. In probabilistic program analysis, moments come symbolically and sampling unbounded loops is costly; Inference of Probabilistic Programs with Moment-Matching Gaussian Mixtures reports Pyro at about 9 s and Bayesian Logic (BLOG), a probabilistic programming language, at about 5 s against 0.192 s for the Gaussian-mixture method on a 100-node radar model. Estimating tail probabilities near $10^{-6}$ by simulation needs sample counts of order $10^{6}$ or more.
Proposed Experiments
Experiment 1 runs on synthetic generalized Gaussians with $p\in\{2,1.5,1,0.8,0.5,0.3\}$ and on log-normals. Each comparison pits two hypotheses differing in scale or shape. We plot $E^\star$ against $K$ for each statistic family. The decisive check is a plateau gap above the Bayes error for integer moments when $p<1$, with no gap when $p\ge1$, and with fractional or log-moments reaching the Bayes error within a few constraints.
Experiment 2 is the finite-sample version on the same families plus the Actitracker accelerometer data used by the robust-testing paper. Activity signals in that data have visibly non-Gaussian magnitude distributions. We report error against $n$ and the sample size required for 5% error.
Experiment 3 targets moment-only verification. It uses the probabilistic-loop benchmarks from the K-series paper whose outputs become multiplicative or heavy-tailed at large iteration counts: random walks and the Vasicek model. We compute exact symbolic moments and ask a tail-threshold question, namely whether $P(X>t)$ exceeds $\alpha$. We compare three bounds against ground truth from long simulation: the worst-case bounds implied by integer moments, those implied by alternative moments, and the single answers given by maximum entropy and K-series.
The proposal is falsified if the integer-moment penalty is negligible at practical $K$. It is also falsified if log-moments and fractional moments both fail to reduce it. The first outcome would mean that existing moment-based tools are safe to use in heavy-tailed settings, which would itself be worth reporting.
References
- Moment Problem in Replica Method
- Robust Multi-Hypothesis Testing with Moment-Constrained Uncertainty Sets
- Analytical properties of generalized Gaussian distributions
- Constructive Approximation under Carleman's Condition, with Applications to Smoothed Analysis
- Distribution Estimation for Probabilistic Loops
- Moment-based Density Elicitation with Applications in Probabilistic Loops
- Inference of Probabilistic Programs with Moment-Matching Gaussian Mixtures