Abstract
Robust estimation methods for the mean vector, scatter matrix, and covariance matrix are essential when dealing with multivariate heavy-tailed distributions. The fitHeavyTail R package implements advanced algorithms for distributions such as angular Gaussian (via Tyler's method), Cauchy, and Student's $t$, including the multivariate skewed $t$ distribution. By utilizing Expectation-Maximization (EM) and Majorization-Minimization (MM) frameworks, the package provides reliable statistical estimates even in the presence of outliers, missing data, and high-dimensionality. This package consolidates research from 2014 to 2021 into a single, efficient interface.
Introduction
In classical statistics, data is often assumed to follow a Gaussian (Normal) distribution. However, real-world data in fields such as finance, biology, and signal processing frequently exhibit "heavy tails," where extreme values (outliers) occur much more often than a Normal distribution would predict. Standard estimators, such as the sample mean and sample covariance, are highly sensitive to these outliers, often leading to biased or nonsensical results.
Furthermore, real-world datasets are often "messy," containing missing values (NAs) or having more variables than observations (high-dimensionality). fitHeavyTail addresses these challenges by providing a unified, computationally efficient framework for robust estimation under non-normal conditions.
Methods and Implementation
The package supports several sophisticated distributional models, including multivariate Cauchy, Student’s $t$, and the more complex Skewed $t$ distributions. It also incorporates Tyler’s M-estimator, which is "distribution-free" within the family of elliptical distributions.
To find Maximum Likelihood Estimates (MLE), the package employs iterative optimization frameworks like Expectation-Maximization (EM) and Majorization-Minimization (MM). These methods are designed to find optimal parameters efficiently, even when the underlying likelihood functions are complex.
As shown in the parameter definitions above, users have fine-grained control over the estimation process, including setting initial values for parameters like mu and scatter, and defining convergence criteria through maximum iterations (max_iter) and relative tolerances (ptol, ftol).
Key Features and Insights
- The $\nu$ Problem: Estimating the "degrees of freedom" ($\nu$) is a significant challenge in fitting Student’s $t$ distributions. The package introduces novel POP (Power-of-the-Projected) and OPP (Optimal-Power-Projected) estimators based on Random Matrix Theory, which provide much more stable estimates of $\nu$ than traditional methods.
- Scatter vs. Covariance: The package carefully distinguishes between the scatter matrix (a parameter of the distribution shape) and the covariance matrix (which may not exist for very heavy-tailed distributions like Cauchy). It provides methods to scale scatter matrices into valid covariance estimates when appropriate.
- Factor Models: For high-dimensional data where a full covariance matrix might be too noisy, the package allows users to specify a factor model structure.
- Missing Data Handling: Unlike many robust estimators that require complete datasets, these implementations can handle data containing NAs, performing estimation directly from available observations.
Key Results and Package Outputs
The fitHeavyTail package serves as a "one-stop-shop" for robust estimation. The functions are significantly more resilient to outliers than standard R functions like mean() and cov(). By using MM and EM frameworks, the package ensures that estimated covariance matrices remain positive semi-definite, a critical requirement for downstream tasks like portfolio optimization.
The output of the estimation functions is a comprehensive list containing parameter estimates (mean, covariance, scatter) and diagnostic information such as convergence status, iteration count, and computational time. If requested, the package can also return a record of parameter values at each iteration.
Conclusion
While iterative EM/MM algorithms are more computationally expensive than simple sample statistics, the gains in numerical stability and resilience to outliers make fitHeavyTail an essential tool for modern statistical analysis. Future directions include the ongoing refinement of POP/OPP estimators for even better stability in extreme high-dimensional settings where variables vastly outnumber samples.
Summary updated