Abstract
We consider small factor analysis models with one or two factors. Fixing the number of factors, we prove a finiteness result about the covariance matrix parameter space when the size of the covariance matrix increases. According to this result, there exists a distinguished matrix size starting at which one can determine whether a given covariance matrix belongs to the parameter space by determining whether all principal submatrices of the distinguished size belong to the corresponding parameter space. We show that the distinguished matrix size is equal to four in the one-factor model and six with two factors.
Introduction
In classical multivariate statistics, the Likelihood Ratio Test (LRT) serves as a standard tool for assessing model fit. However, the validity of the LRT typically hinges on the assumption that the sample size ($n$) exceeds the number of variables ($p$). In the contemporary "Big Data" landscape, researchers frequently encounter high-dimensional datasets where $p > n$. In such scenarios, the sample covariance matrix is singular, rendering traditional tests based on determinants and likelihood ratios inapplicable.
This paper provides a theoretical foundation for addressing this challenge by exploring the "finiteness" of factor analysis models. The authors investigate whether the global membership of a large covariance matrix in a model can be determined by examining its smaller, local sub-components.
The Factor Analysis Model
The authors define the factor analysis model $F_{p,m}$ as the set of $p \times p$ positive definite covariance matrices $\Sigma$ that can be decomposed into a diagonal matrix and a low-rank matrix:
$$\Sigma = \Psi + \Lambda \Lambda^T$$
In this decomposition, $\Psi$ is a diagonal matrix with positive entries representing unique variance or noise, and $\Lambda$ is a $p \times m$ matrix of factor loadings representing $m$ shared latent factors. The rank of the component $\Lambda \Lambda^T$ is at most $m$.
Local-to-Global Consistency
The core contribution of the work is the identification of a "distinguished size" $k$. If every $k \times k$ principal submatrix of a $p \times p$ matrix $\Sigma$ belongs to the model $F_{k,m}$, then the entire matrix $\Sigma$ must belong to $F_{p,m}$. This property suggests that the geometric and algebraic structure of the model is "encoded" into its smallest constituent parts.
The One-Factor Case ($m=1$)
For a single-factor model, the authors prove (Theorem 4) that the distinguished size is four. A positive definite matrix belongs to the 1-factor model if and only if every $4 \times 4$ principal submatrix belongs to the 1-factor model. This result justifies the development of statistical tests that work even when $p > n$ by focusing on these small-block properties.
The Two-Factor Case ($m=2$)
The complexity increases with additional factors. For the two-factor model, the authors establish (Theorem 7) that the distinguished size is six. They demonstrate that while $5 \times 5$ submatrices are insufficient to guarantee global fit (as shown in Example 8), $6 \times 6$ submatrices provide the necessary information for a definitive determination.
Methodology: Algebraic Statistics
The research utilizes tools from algebraic statistics and linear algebra to characterize the parameter space. Key elements of the proof include:
- Matrix Minors: The authors use the properties of determinants of submatrices (minors) to characterize the rank of the latent factor component.
- Inductive Proofs: The results are established by proving that if the property holds for a matrix of size $p-1$, it can be extended to size $p$.
- Uniqueness of Decomposition: A critical component of the proof is establishing conditions under which the decomposition of $\Sigma$ into noise and factors is unique, which is essential for the induction to hold.
Implications and Limitations
The findings suggest a general pattern where the distinguished size is $2(m+1)$. While the authors conjecture this holds for $m \ge 3$, the inductive proof method becomes significantly more complex due to the algebraic requirements of the uniqueness lemmas and the behavior of vanishing minors in sparse matrices.
From a practical standpoint, this work provides the mathematical justification for developing statistical tests that aggregate information from small sub-blocks of variables. While the paper does not derive the specific distribution theory for implementation, it establishes the geometric boundaries necessary for such "statistical engineering" in high-dimensional inference.
Summary updated