Abstract
In this paper, we propose an alternative proof for the uniqueness of Maronna's $M$-estimator of scatter (Maronna, 1976) for $N$ vector observations $\mathbf y_1,...,\mathbf y_N\in\mathbb R^m$ under a mild constraint of linear independence of any subset of $m$ of these vectors. This entails in particular almost sure uniqueness for random vectors $\mathbf y_i$ with a density as long as $N>m$. This approach allows to establish further relations that demonstrate that a properly normalized Tyler's $M$-estimator of scatter (Tyler, 1987) can be considered as a limit of Maronna's $M$-estimator. More precisely, the contribution is to show that each $M$-estimator converges towards a particular Tyler's $M$-estimator. These results find important implications in recent works on the large dimensional (random matrix) regime of robust $M$-estimation.
Introduction
Robust statistics aims to estimate the "scatter" of data—a generalization of covariance—in a manner resilient to outliers. Maronna’s $M$-estimator, introduced in 1976, remains a foundational tool for this purpose. However, for decades, the mathematical guarantee of its uniqueness relied on a restrictive condition: the ratio of the number of features ($m$) to the number of observations ($N$) had to be very small.
In modern high-dimensional contexts, where $m$ and $N$ are comparable (the "large dimensional regime"), these classical proofs often fail. This research provides a rigorous mathematical proof for the uniqueness of Maronna’s $M$-estimator under significantly broader conditions, specifically requiring only $N > m$ for most practical datasets.
Methodology
The authors utilize an analytical approach centered on optimization within the manifold of symmetric positive definite matrices ($PSD_m$).
Admissible Families and Objective Functions
The proof introduces the concept of an admissible family, where any subset of $m$ vectors from the $N$ observations is linearly independent. This condition is generally satisfied by real-world continuous data. A positive functional $H(t, \cdot)$ is constructed such that its critical points correspond to the solutions of the $M$-estimator's implicit equation.
Topological Proof of Uniqueness
To establish uniqueness, the authors demonstrate that the functional $H(t, \cdot)$ is uniformly bounded and vanishes at the boundaries of the $PSD_m$ cone. By applying the Mountain Pass Theorem, they show that if multiple local maxima existed, a saddle point would necessarily exist between them. Analysis of the Hessian reveals that all critical points must be strict local maxima, thereby precluding the existence of a saddle point and guaranteeing a unique solution.
Results and Convergence
The core theoretical result (Theorem II.3) proves that for any admissible family and a properly defined weight function, a unique solution $M(t)$ exists for all $t > 0$.
Relation to Tyler's Estimator
A significant contribution of this work is establishing a formal link between Maronna’s and Tyler’s $M$-estimators. The parameter $t$ acts as a "slider": as $t \to 0$, the Maronna estimator $M(t)$ converges to a scaled version of Tyler’s distribution-free $M$-estimator. This identifies Tyler’s estimator as a limiting case within the Maronna family.
The convergence is illustrated in Figure 1, which shows the behavior of a distance metric $C(t)$ as $t$ approaches zero. The simulation used synthetic Gaussian data with a Toeplitz covariance matrix (correlation $\rho^{|i-j|}$) in a high-dimensional regime ($N = m + 1 = 51$), demonstrating that the estimator remains valid even when traditional uniqueness conditions are violated.
Conclusion
This work demonstrates that Maronna’s estimator is more robust and stable than previously understood, requiring only $N > m$ for uniqueness. Future research may extend these proofs to scenarios where both location and scatter must be estimated simultaneously, or explore even less restrictive weight function constraints.
Summary updated