Large Covariance Matrix Estimation With Oracle Statistical Rate via Majorization-Minimization

Quan Wei, Ziping ZhaoIEEE TSP 2023 · OpenReview

Abstract

The <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\boldsymbol{\ell}_{\mathbf{1}}$</tex-math></inline-formula> penalized covariance estimator has been widely used for estimating large sparse covariance matrices. It is recognized that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\boldsymbol{\ell}_{\mathbf{1}}$</tex-math></inline-formula> penalty introduces a non-negligible estimation bias, while a proper utilization of non-convex penalty may lead to an estimator with a refined statistical rate of convergence. To eliminate the estimation bias, in this paper we propose to estimate large sparse covariance matrices using the non-convex penalty. It is challenging to analyze the theoretical properties of the resulting estimator because popular iterative algorithms for convex optimization no longer have global convergence guarantees for non-convex optimization. To tackle this issue, an efficient algorithm based on the majorization-minimization (MM) framework is developed by solving a sequence of convex relaxation subproblems. An approximation solution to each subproblem is obtained via the proximal gradient method with a linear convergence rate. We clearly establish the statistical properties of all the approximate solutions generated by the MM-based algorithm and prove that the proposed estimator achieves the oracle statistical rate in the Frobenius norm under weak technical assumptions. We also consider a modification of the proposed estimation method using the correlation matrix and show that the modified correlation-based covariance estimator enjoys a better rate in the spectral norm. Our theoretical findings are corroborated through extensive numerical experiments on both synthetic data and real-world datasets.
BibTeX
@article{wei2023large,
  author = {Quan Wei and Ziping Zhao},
  title = {{Large Covariance Matrix Estimation With Oracle Statistical Rate via Majorization-Minimization}},
  journal = {IEEE Trans. Signal Process.},
  year = {2023},
  url = {https://openreview.net/forum?id=bmv91uk4tM},
}

Loading related content...

Figure