A Toeplitz-Tyler Estimation of the Model Order in Large Dimensional Regime

Eugénie Terreaux, Jean‐Philippe Ovarlez, Frédéric Pascal2018 · DOI

TL;DR

This paper introduces a novel, robust algorithm for estimating the number of sources in a complex signal corrupted by correlated, non-Gaussian noise, particularly in high-dimensional settings. The method leverages a combination of Toeplitz matrix structure and Tyler M-estimators, simplifying the model order selection process by relying on the well-established Marchenko-Pastur law.

What Problem Does This Paper Address?

Estimating the number of underlying sources within a signal is a fundamental challenge in many signal processing applications, from radar to communications. Traditional methods often rely on accurate estimation of the signal's covariance matrix. However, these methods can struggle when the noise is not only correlated but also non-Gaussian, and when the dimensionality of the data (number of sensors or observations) becomes very large. Existing robust methods for non-Gaussian noise often require complex, difficult-to-derive thresholds or assume known covariance matrices. This paper aims to bridge this gap by providing a robust and computationally tractable method for model order selection in the presence of correlated, complex elliptically symmetric (CES) noise in a large dimensional regime.

Methods

The proposed algorithm consists of two main steps:

  1. Signal Whitening: The first crucial step is to whiten the signal. This is achieved by estimating the noise's scatter matrix, denoted by $C$. The authors propose a Toeplitz-enforced Tyler M-estimator for $C$. This approach leverages the Toeplitz structure of the true scatter matrix and the robustness of Tyler M-estimators, which are known to perform well for non-Gaussian data. The consistency of this estimator is theoretically established.
  2. Model Order Estimation: After whitening the signal using the estimated scatter matrix, the problem reduces to analyzing the eigenvalues of the whitened signal's scatter matrix. The authors show that the distribution of these eigenvalues converges to the Marchenko-Pastur distribution in the large dimensional regime. This is a significant result, as the Marchenko-Pastur law provides a well-defined spectral support for random matrices. The number of sources is then estimated by counting the number of eigenvalues that fall outside the support of the Marchenko-Pastur law, using its right edge, $(1+\sqrt{c})^2$, as a threshold. This approach avoids the need for complicated, data-dependent thresholds previously required by some robust methods.

Datasets

The paper does not introduce new datasets. Instead, it relies on simulated data to evaluate the performance of the proposed algorithm. The simulations involve generating signals with correlated CES noise and varying Signal-to-Noise Ratios (SNR) to test the algorithm's effectiveness.

Key Insights and Ideas

  • Robustness to Non-Gaussian and Correlated Noise: The combination of Tyler M-estimators and the Toeplitz structure allows for robust estimation even when the noise is non-Gaussian and correlated, a scenario where many standard methods fail.
  • Simplification of Thresholding: A major contribution is the elimination of complex, a priori-dependent thresholds. By leveraging the Marchenko-Pastur law, the algorithm uses a universal threshold derived from the noise statistics, making it more practical.
  • Large Dimensionality Advantage: The paper explicitly addresses the "large dimensional regime" where the number of observations and the dimension of the data grow proportionally. This is a common scenario in modern sensing systems, and the proposed method is designed to exploit this regime.
  • Toeplitz Structure Exploitation: Incorporating the Toeplitz structure of the noise covariance matrix into the estimation process leads to more consistent and efficient estimates, especially in correlated noise scenarios.

Key Results

The paper demonstrates through simulations that the proposed algorithm, termed TylTER, can accurately estimate the number of sources across a range of SNRs.

  • Accurate Source Counting: Figures 4 and 5 show that TylTER correctly identifies the true number of sources (p) for SNRs as low as 20 dB, outperforming the AIC method, which tends to overestimate.
  • Comparison with Prior Work: TylTER is compared to a previously proposed robust method (FPTER) and the AIC. TylTER consistently performs well, often detecting sources at lower SNRs than FPTER, especially when the noise distribution deviates from the assumptions made by FPTER.
  • Robustness to Noise Distribution: The simulations show that TylTER remains effective for both inverse gamma and Student-t distributed noise, highlighting its robustness to different CES noise characteristics.

Limitations and Future Work

  • Computational Complexity: While the method simplifies thresholding, the Tyler M-estimator itself can be computationally intensive, especially for very high dimensions. The paper mentions that the convergence rate of the Toeplitz-enforced Tyler estimator can be slow and suggests banding or tapering as potential enhancements, which could be explored further.
  • Assumptions: The method relies on the assumption of correlated CES noise and a large dimensional regime. Its performance in scenarios with different noise models or low-dimensional data is not explicitly discussed.
  • Future Work: The authors suggest that the promising results encourage the application of this method to a wider range of signals and scenarios. Further research could focus on optimizing the computational aspects and exploring its applicability to other signal processing tasks beyond model order selection.

Summary updated

Abstract

International audience
BibTeX
@misc{terreaux2018toeplitz,
  author = {Eugénie Terreaux and Jean‐Philippe Ovarlez and Frédéric Pascal},
  title = {{A Toeplitz-Tyler Estimation of the Model Order in Large Dimensional Regime}},
  year = {2018},
  doi = {10.1109/icassp.2018.8461915},
}

Loading related content...

Figure