Improved estimation of the degree of freedom parameter of multivariate $t$-distribution

Frédéric Pascal, Esa Ollila, Daniel P. Palomar2021 · DOI

TL;DR

This paper introduces a novel and improved method for estimating the degree of freedom (d.o.f.) parameter of the multivariate $t$-distribution. This parameter is crucial for modeling heavy-tailed data, and the proposed estimator, called $\hat{\nu}_{\text{POP}}$, demonstrates superior performance compared to existing methods, especially in challenging high-dimensional and heavy-tailed scenarios.

What Problem Does This Paper Address?

The multivariate $t$-distribution (MVT) is a powerful tool for modeling data with heavy tails, which are common in fields like finance and signal processing. A key characteristic of the MVT distribution is its degree of freedom (d.o.f.) parameter, denoted by $\nu$. This parameter controls the "thickness" of the tails: a smaller $\nu$ indicates heavier tails, while as $\nu$ approaches infinity, the MVT distribution converges to the multivariate Gaussian distribution. Accurately estimating $\nu$ is critical for correctly modeling the data's tail behavior. However, existing methods for estimating $\nu$ can be unstable or perform poorly, particularly in high-dimensional settings or when the data exhibits very heavy tails. This paper aims to address this challenge by developing a more robust and accurate estimator for $\nu$.

Methods

The core of the proposed methodology lies in a novel relationship between the scatter matrix and the covariance matrix of the MVT distribution. The authors derive an estimator for $\nu$ based on the expectation of a specific quadratic form related to the Mahalanobis distance. This approach is particularly well-suited for the "random matrix theory" (RMT) regime, where both the number of observations ($n$) and the dimension of the data ($p$) tend to infinity, with their ratio $p/n$ converging to a constant $c \in [0, 1)$.

The paper outlines an iterative algorithm (Algorithm 1) for computing the proposed estimator, $\hat{\nu}_{\text{POP}}$. This algorithm involves iteratively updating estimates of the scatter matrix and the d.o.f. parameter. A key step involves using an M-estimator of scatter, computed by excluding one observation at a time, to estimate the expectation of the Mahalanobis distance. The theoretical justification for the estimator's convergence is provided, notably without relying on the classical independence assumption, which is often violated in practice.

Datasets

Not discussed in the paper. The evaluation is performed using simulated data.

Key Insights and Ideas

  • Bridging Scatter and Covariance: The paper establishes a crucial link between the scatter matrix and the covariance matrix in the MVT distribution, which is fundamental to their estimation strategy.
  • RMT Regime Advantage: The proposed estimator is specifically designed and theoretically justified for the random matrix theory (RMT) regime ($n, p \to \infty$, $p/n \to c$), where many traditional methods struggle.
  • Robustness to Independence Violations: The estimator's convergence is proven even when the standard independence assumption is not met, making it more practical for real-world data.
  • Improved Performance in Heavy Tails: Simulations show that $\hat{\nu}_{\text{POP}}$ significantly outperforms existing methods, especially when dealing with very heavy-tailed data ($\nu=3$).
  • Consistent Performance: Even as the data distribution approaches Gaussianity ($\nu=10$), the proposed estimator maintains strong performance, unlike some other methods that degrade.

Key Results

The paper presents simulation results demonstrating the superiority of the proposed estimator, $\hat{\nu}_{\text{POP}}$, over two benchmark methods: a Kurtosis-based estimator ($\hat{\nu}_\kappa$) and a previously proposed iterative estimator ($\hat{\nu}^{(0)}$).

  • Heavy-tailed scenarios ($\nu=3$): $\hat{\nu}_{\text{POP}}$ shows a substantial improvement in Mean Square Error (MSE) compared to both $\hat{\nu}_\kappa$ and $\hat{\nu}^{(0)}$. The improvement is particularly pronounced for smaller sample sizes and dimensions.
  • Near-Gaussian scenarios ($\nu=10$): While $\hat{\nu}_\kappa$ performs well, $\hat{\nu}^{(0)}$'s performance degrades. $\hat{\nu}_{\text{POP}}$ consistently provides excellent estimation accuracy across various settings.
  • Impact of Dimension ($p$): Increasing the dimension $p$ generally improves estimation accuracy for all methods, but $\hat{\nu}_{\text{POP}}$ consistently leads in performance.
  • General Conclusion: The authors conclude that $\hat{\nu}_{\text{POP}}$ offers the best estimation performance, with significant gains observed when both $n$ and $p$ are small and of similar order.

Limitations and Future Work

  • Statistical Analysis: The authors mention that future work will focus on the statistical analysis of the proposed estimator.
  • Real-World Data: Experimentations on real data, particularly in finance, are suggested as a future direction.
  • RMT Regime Focus: While the estimator is designed for the RMT regime, its performance in very low-dimensional settings ($p \ll n$) is not explicitly detailed, though simulations suggest good performance even when $n$ increases.

Summary updated

Abstract

The multivariate <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$t$</tex> (MVT)-distribution is a widely used statistical model in various application domains, mainly due to its adaptability to heavy-tailed data. However, estimating the degree of freedom (d.o.f) parameter, that controls the shape of the distribution, remains a challenging problem. In this work, we develop a novel methodology and design various algorithms for estimating the d.o <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{f}$</tex> parameter. More precisely, based on a key relationship between scatter and covariance matrices for the t-distribution, the estimator is derived from the expectation of a particular quadratic form and is proved to converge although the classical independence assumption is not fulfilled. finally, some preliminary simulations show the improvement of the proposed approach with respect to state-of-the-art methods.
BibTeX
@inproceedings{pascal2021improved,
  author = {Frédéric Pascal and Esa Ollila and Daniel P. Palomar},
  title = {{Improved estimation of the degree of freedom parameter of multivariate $t$-distribution}},
  booktitle = {2021},
  year = {2021},
  doi = {10.23919/eusipco54536.2021.9616162},
}
Figure