TL;DR
This paper bridges a critical gap in understanding Tyler's estimator (TyE), a robust method for estimating covariance matrices, especially in non-Gaussian settings. The authors derive new theoretical properties that show TyE behaves similarly to the simpler, well-understood Sample Covariance Matrix (SCM) under certain conditions, making TyE more amenable to analysis and application in signal processing.
What Problem Does This Paper Address?
Estimating the covariance matrix (CM) is fundamental in many signal processing tasks like filtering, detection, and classification. The standard approach is the Sample Covariance Matrix (SCM). While well-behaved for Gaussian data, the SCM's performance degrades significantly when data deviates from Gaussianity or contains outliers. Robust estimators, like Tyler's estimator (TyE), are designed to handle these non-Gaussian scenarios. However, TyE's complex, non-explicit form has made its statistical properties difficult to fully grasp, hindering its widespread adoption despite its practical utility. This paper aims to demystify TyE by providing a theoretical framework to approximate its behavior with the more familiar SCM, particularly for complex elliptically symmetric (CES) distributions.
Methods
The core of this paper is the derivation of the asymptotic distribution of the difference between Tyler's estimator (TyE) and the Sample Covariance Matrix (SCM) for complex elliptically symmetric (CES) distributed data. The authors leverage existing results on the asymptotic properties of both TyE and the SCM, particularly their behavior under Gaussian assumptions. They then mathematically analyze the difference between these two estimators, showing that under specific conditions (large sample sizes and dimensions), their asymptotic distributions converge. This convergence allows for the approximation of TyE's behavior by the SCM's behavior, which is well-characterized by the Wishart distribution. The proof involves detailed algebraic manipulations of matrix operations and expectation calculations, utilizing properties of complex random vectors and their distributions.
Datasets
Not discussed in the paper. The simulations use synthetic data generated according to specified distributions (complex Gaussian).
Key Insights and Ideas
- TyE's Approximation by SCM: The most significant finding is that Tyler's estimator, despite its complex form, asymptotically behaves like the Sample Covariance Matrix (SCM) for complex elliptically symmetric distributions. This is a crucial theoretical justification for using TyE in practice and leveraging the well-established properties of the SCM (like the Wishart distribution) for analysis.
- Faster Convergence of TyE: The paper highlights that the matrices characterizing the asymptotic difference between TyE and SCM have elements that are smaller than those characterizing TyE's deviation from the true covariance matrix. This implies that TyE approaches the SCM's behavior faster than it approaches the true covariance matrix, especially in high-dimensional settings.
- Dimension Matters: The study shows that as the dimension of the data increases, the difference between TyE and SCM diminishes. This makes TyE a particularly attractive option for high-dimensional signal processing problems where SCM performance typically degrades.
- Distribution-Free Advantage: The paper emphasizes that Tyler's estimator is "distribution-free" over the class of CES distributions. This means its robust properties are not tied to a specific distribution within this broad class, making it versatile.
Key Results
The paper's main theoretical result is the derivation of the asymptotic distribution of the difference between TyE and SCM: $\sqrt{N}\text{vec}(\hat{M}_{\text{TyE}} - \hat{M}_{\text{SCM}}) \xrightarrow{d} \mathcal{CN}(0, \Sigma, \Omega)$. The matrices $\Sigma$ and $\Omega$ are explicitly derived and shown to depend on the true covariance matrix $M$.
Simulation results validate these theoretical findings:
- Figures 1 and 2 show that the empirical errors between the estimated and theoretical values of the covariance matrix elements decrease as the number of samples ($N$) increases, confirming the asymptotic nature of the results.
- Figure 2 also demonstrates that as the data dimension ($m$) increases, the error elements tend towards zero, supporting the idea that TyE's behavior becomes increasingly similar to SCM's in high dimensions.
- Figure 3 shows good agreement between empirical and theoretical values for correlated data, further validating the derived formulas.
Limitations and Future Work
- Assumptions: The derivations rely on the data belonging to the class of complex elliptically symmetric (CES) distributions. While broad, this class does not encompass all possible non-Gaussian distributions.
- Approximation: The paper provides an asymptotic approximation. The accuracy of this approximation for finite sample sizes and dimensions is explored through simulations, but the exact bounds are not detailed.
- Future Work: The authors suggest that this work allows for better understanding and theoretical adjustment of parameters (like detection thresholds) in signal processing applications, implying future work could focus on specific applications where TyE's properties can be directly leveraged. The paper also hints at the potential for using these results in robust estimation contexts beyond CES distributions.
Summary updated