Uniqueness of Maronna's $M$-estimators of scatter

Yacine Chitour, Romain Couillet, Frédéric Pascal

Abstract

In this paper, we propose an alternative proof for the uniqueness of Maronna's $M$-estimator of scatter (Maronna, 1976) for $N$ vector observations $\mathbf y_1,...,\mathbf y_N\in\mathbb R^m$ under a mild constraint of linear independence of any subset of $m$ of these vectors. This entails in particular almost sure uniqueness for random vectors $\mathbf y_i$ with a density as long as $N>m$. This approach allows to establish further relations that demonstrate that a properly normalized Tyler's $M$-estimator of scatter (Tyler, 1987) can be considered as a limit of Maronna's $M$-estimator. More precisely, the contribution is to show that each $M$-estimator converges towards a particular Tyler's $M$-estimator. These results find important implications in recent works on the large dimensional (random matrix) regime of robust $M$-estimation.

Introduction

Robust statistics aims to estimate the "scatter" of data—a generalization of covariance—in a manner resilient to outliers. Maronna’s $M$-estimator, introduced in 1976, remains a foundational tool for this purpose. However, for decades, the mathematical guarantee of its uniqueness relied on a restrictive condition: the ratio of the number of features ($m$) to the number of observations ($N$) had to be very small.

In modern high-dimensional contexts, where $m$ and $N$ are comparable (the "large dimensional regime"), these classical proofs often fail. This research provides a rigorous mathematical proof for the uniqueness of Maronna’s $M$-estimator under significantly broader conditions, specifically requiring only $N > m$ for most practical datasets.

Methodology

The authors utilize an analytical approach centered on optimization within the manifold of symmetric positive definite matrices ($PSD_m$).

Admissible Families and Objective Functions

The proof introduces the concept of an admissible family, where any subset of $m$ vectors from the $N$ observations is linearly independent. This condition is generally satisfied by real-world continuous data. A positive functional $H(t, \cdot)$ is constructed such that its critical points correspond to the solutions of the $M$-estimator's implicit equation.

Topological Proof of Uniqueness

To establish uniqueness, the authors demonstrate that the functional $H(t, \cdot)$ is uniformly bounded and vanishes at the boundaries of the $PSD_m$ cone. By applying the Mountain Pass Theorem, they show that if multiple local maxima existed, a saddle point would necessarily exist between them. Analysis of the Hessian reveals that all critical points must be strict local maxima, thereby precluding the existence of a saddle point and guaranteeing a unique solution.

Results and Convergence

The core theoretical result (Theorem II.3) proves that for any admissible family and a properly defined weight function, a unique solution $M(t)$ exists for all $t > 0$.

Relation to Tyler's Estimator

A significant contribution of this work is establishing a formal link between Maronna’s and Tyler’s $M$-estimators. The parameter $t$ acts as a "slider": as $t \to 0$, the Maronna estimator $M(t)$ converges to a scaled version of Tyler’s distribution-free $M$-estimator. This identifies Tyler’s estimator as a limiting case within the Maronna family.

Convergence of Maronna's $M$-estimator to Tyler's $M$-estimator as $t \to 0$ for different correlation levels $\rho$.

The convergence is illustrated in Figure 1, which shows the behavior of a distance metric $C(t)$ as $t$ approaches zero. The simulation used synthetic Gaussian data with a Toeplitz covariance matrix (correlation $\rho^{|i-j|}$) in a high-dimensional regime ($N = m + 1 = 51$), demonstrating that the estimator remains valid even when traditional uniqueness conditions are violated.

Conclusion

This work demonstrates that Maronna’s estimator is more robust and stable than previously understood, requiring only $N > m$ for uniqueness. Future research may extend these proofs to scenarios where both location and scatter must be estimated simultaneously, or explore even less restrictive weight function constraints.

Summary updated

Abstract

In this paper, {we propose an alternative proof for the uniqueness} of Maronna's $M$-estimator of scatter (Maronna, 1976) for $N$ vector observations $\mathbf y_1,...,\mathbf y_N\in\mathbb R^m$ under a mild constraint of linear independence of any subset of $m$ of these vectors. This entails in particular almost sure uniqueness for random vectors $\mathbf y_i$ with a density as long as $N>m$. {This approach allows to establish further relations that demonstrate that a properly normalized Tyler's $M$-estimator of scatter (Tyler, 1987) can be considered as a limit of Maronna's $M$-estimator. More precisely, the contribution is to show that each $M$-estimator converges towards a particular Tyler's $M$-estimator.} These results find important implications in recent works on the large dimensional (random matrix) regime of robust $M$-estimation.
BibTeX
@article{chitour2015uniqueness,
  author = {Yacine Chitour and Romain Couillet and Frédéric Pascal},
  title = {{Uniqueness of Maronna's $M$-estimators of scatter}},
  journal = {IEEE Signal Processing Letters},
  year = {2015},
  doi = {10.1109/lsp.2014.2367547},
}
Figure