fitHeavyTail: Mean and Covariance Matrix Estimation under Heavy Tails

Daniel P. Palomar, Rui Zhou, Xiwen WangOpenalex Unknown 2019 · DOI · OpenAlex

Abstract

Robust estimation methods for the mean vector, scatter matrix, and covariance matrix are essential when dealing with multivariate heavy-tailed distributions. The fitHeavyTail R package implements advanced algorithms for distributions such as angular Gaussian (via Tyler's method), Cauchy, and Student's $t$, including the multivariate skewed $t$ distribution. By utilizing Expectation-Maximization (EM) and Majorization-Minimization (MM) frameworks, the package provides reliable statistical estimates even in the presence of outliers, missing data, and high-dimensionality. This package consolidates research from 2014 to 2021 into a single, efficient interface.

Introduction

In classical statistics, data is often assumed to follow a Gaussian (Normal) distribution. However, real-world data in fields such as finance, biology, and signal processing frequently exhibit "heavy tails," where extreme values (outliers) occur much more often than a Normal distribution would predict. Standard estimators, such as the sample mean and sample covariance, are highly sensitive to these outliers, often leading to biased or nonsensical results.

Furthermore, real-world datasets are often "messy," containing missing values (NAs) or having more variables than observations (high-dimensionality). fitHeavyTail addresses these challenges by providing a unified, computationally efficient framework for robust estimation under non-normal conditions.

Methods and Implementation

The package supports several sophisticated distributional models, including multivariate Cauchy, Student’s $t$, and the more complex Skewed $t$ distributions. It also incorporates Tyler’s M-estimator, which is "distribution-free" within the family of elliptical distributions.

To find Maximum Likelihood Estimates (MLE), the package employs iterative optimization frameworks like Expectation-Maximization (EM) and Majorization-Minimization (MM). These methods are designed to find optimal parameters efficiently, even when the underlying likelihood functions are complex.

Table 1: Input parameters for the iterative estimation methods in fitHeavyTail.

As shown in the parameter definitions above, users have fine-grained control over the estimation process, including setting initial values for parameters like mu and scatter, and defining convergence criteria through maximum iterations (max_iter) and relative tolerances (ptol, ftol).

Key Features and Insights

  • The $\nu$ Problem: Estimating the "degrees of freedom" ($\nu$) is a significant challenge in fitting Student’s $t$ distributions. The package introduces novel POP (Power-of-the-Projected) and OPP (Optimal-Power-Projected) estimators based on Random Matrix Theory, which provide much more stable estimates of $\nu$ than traditional methods.
  • Scatter vs. Covariance: The package carefully distinguishes between the scatter matrix (a parameter of the distribution shape) and the covariance matrix (which may not exist for very heavy-tailed distributions like Cauchy). It provides methods to scale scatter matrices into valid covariance estimates when appropriate.
  • Factor Models: For high-dimensional data where a full covariance matrix might be too noisy, the package allows users to specify a factor model structure.
  • Missing Data Handling: Unlike many robust estimators that require complete datasets, these implementations can handle data containing NAs, performing estimation directly from available observations.

Key Results and Package Outputs

The fitHeavyTail package serves as a "one-stop-shop" for robust estimation. The functions are significantly more resilient to outliers than standard R functions like mean() and cov(). By using MM and EM frameworks, the package ensures that estimated covariance matrices remain positive semi-definite, a critical requirement for downstream tasks like portfolio optimization.

Table 2: Elements returned by the estimation functions, providing a comprehensive summary of the model fit.

The output of the estimation functions is a comprehensive list containing parameter estimates (mean, covariance, scatter) and diagnostic information such as convergence status, iteration count, and computational time. If requested, the package can also return a record of parameter values at each iteration.

Conclusion

While iterative EM/MM algorithms are more computationally expensive than simple sample statistics, the gains in numerical stability and resilience to outliers make fitHeavyTail an essential tool for modern statistical analysis. Future directions include the ongoing refinement of POP/OPP estimators for even better stability in extreme high-dimensional settings where variables vastly outnumber samples.

Summary updated

Abstract

Robust estimation methods for the mean vector, scatter matrix, and covariance matrix (if it exists) from data (possibly containing NAs) under multivariate heavy-tailed distributions such as angular Gaussian (via Tyler's method), Cauchy, and Student's t distributions. Additionally, a factor model structure can be specified for the covariance matrix. The latest revision also includes the multivariate skewed t distribution. The package is based on the papers: Sun, Babu, and Palomar (2014); Sun, Babu, and Palomar (2015); Liu and Rubin (1995); Zhou, Liu, Kumar, and Palomar (2019); Pascal, Ollila, and Palomar (2021).
BibTeX
@inproceedings{palomar2019fitheavytail,
  author = {Daniel P. Palomar and Rui Zhou and Xiwen Wang},
  title = {{fitHeavyTail: Mean and Covariance Matrix Estimation under Heavy Tails}},
  booktitle = {Openalex Unknown},
  year = {2019},
  doi = {10.32614/cran.package.fitheavytail},
}
Figure