The normalizing constant in multivariate t distributions: dimension one versus higher dimensions

Christophe Ley, Anouk Neven

Introduction

In introductory statistics, the Student $t$-distribution is often introduced as a "heavy-tailed" cousin of the Gaussian (Normal) distribution. As the degrees of freedom ($\nu$) increase, the $t$-distribution converges to the Gaussian. Intuition suggests that because the $t$-distribution has more probability mass in its tails, its mode (the "peak" at the center) must be lower than the Gaussian's to ensure the total area under the curve remains 1. While this holds true in one dimension, researchers Christophe Ley and Anouk Neven have uncovered a surprising reversal in higher dimensions.

The Problem: Does the Peak Always Grow?

While the convergence of the $t$-distribution to the Gaussian is well-established, the nature of this convergence—specifically whether the value at the mode moves monotonically toward the Gaussian value—had not been formally proven across different dimensions ($k$). The authors set out to determine if the peak of the $t$-distribution always increases to meet the Gaussian peak, or if the dimension of the data fundamentally changes this behavior.

Methods: Analyzing the Normalizing Constant

The study focuses on the normalizing constant $c_{\nu,k}$, which represents the value of the $k$-dimensional Student $t$ density at its mode (the origin). Using functional analysis of the Gamma and digamma functions, the authors calculated the derivative of the log of this constant with respect to $\nu$.

The normalizing constant $c_{\nu,k}$ as a function of degrees of freedom $\nu$ for dimensions (a) $k=1$, (b) $k=2$, (c) $k=3$, and (d) $k=4$.

As shown in Figure 1, the behavior of the normalizing constant depends entirely on the dimension $k$. For $k=1$, it increases toward the limit; for $k=2$, it remains perfectly constant; and for $k \ge 3$, it decreases toward the limit.

Key Insights: The Dimension Flip

The most striking finding is this "Dimension Flip":

  • $k = 1$: The peak increases toward the Gaussian limit, matching standard intuition.
  • $k = 2$: The peak height is invariant to the degrees of freedom.
  • $k \ge 3$: The $t$-distribution starts with a higher peak than the Gaussian and decreases as $\nu$ increases.

This "Kurtosis Paradox" occurs because in higher dimensions, the "tails" occupy a massive volume. Even if the $t$-distribution is peakier at the very center, the rapid loss of height just slightly away from the mode is integrated over a much larger domain in high-dimensional space. This allows the distribution to be simultaneously "peakier" at the origin and "heavier" in the far tails.

Key Results: Visualizing the Paradox

The theoretical proofs are supported by density plots that provide a clear visual confirmation of this phenomenon.

Density functions for different degrees of freedom $\nu$ (blue: $\nu=1$, purple: $\nu=2$, gold: $\nu=10$, green: $\nu=\infty$) across dimensions $k=1, 2, 3, 4$.

In Figure 2(d), representing four dimensions, the blue curve (Cauchy distribution, $\nu=1$) towers over the green Gaussian curve ($\nu=\infty$). This directly contradicts the 1D visualization in Figure 2(a), where the Gaussian curve is the tallest. Numerical analysis further confirms this: for $k=4$, a Cauchy distribution puts nearly three times as much probability mass in a tiny ball around the origin compared to a Gaussian distribution.

Conclusion and Future Work

This discovery challenges the "common knowledge" that heavy tails always imply a lower peak. While the mathematical proof is rigorous, the result remains counter-intuitive even to experts. Future research could explore if similar dimension-dependent flips occur in other families of heavy-tailed distributions or how this "peakiness" might influence high-dimensional statistical procedures like Bayesian MAP estimation or mode-seeking algorithms.

Summary updated

Abstract

It is a well-known fact that multivariate Student $t$ distributions converge to multivariate Gaussian distributions as the number of degrees of freedom $ν$ tends to infinity, irrespective of the dimension $k\geq1$. In particular, the Student's value at the mode (that is, the normalizing constant obtained by evaluating the density at the center) $c_{ν,k}=\frac{Γ(\frac{ν+k}{2})}{(πν)^{k/2} Γ( \fracν{2})}$ converges towards the Gaussian value at the mode $c_k=\frac{1}{(2π)^{k/2}}$. In this note, we prove a curious fact: $c_{ν,k}$ tends monotonically to $c_k$ for each $k$, but the monotonicity changes from increasing in dimension $k=1$ to decreasing in dimensions $k\geq3$ whilst being constant in dimension $k=2$. A brief discussion raises the question whether this \emph{a priori} curious finding is a curiosity, \emph{in fine}.
BibTeX
@misc{ley2012normalizing,
  author = {Christophe Ley and Anouk Neven},
  title = {{The normalizing constant in multivariate t distributions: dimension one versus higher dimensions}},
  year = {2012},
  eprint = {1211.1174},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1211.1174},
}
Figure