In causal discovery, we often try to find the "hidden" structure of the world from data we can actually see. The hardest part of this task is dealing with latent variables—unobserved common causes that create correlations between observed variables, making it difficult to tell if $X$ causes $Y$ or if they are both caused by some invisible $Z$.
A powerful line of research has shown that these hidden structures leave "fingerprints" in the statistical properties of our data. Specifically, the number of hidden variables acting as a bottleneck between groups of observed variables manifests as a mathematical constraint on the rank of certain matrices.
The Intuition of the Dimensional Bottleneck
Imagine two groups of people, Group A and Group B. If every person in Group A talks directly to every person in Group B, the "information flow" between them is high-dimensional. However, if all communication between the two groups must pass through a single mediator, the information flow is restricted. No matter how many people are in the groups, the relationship between them is "rank-one" because it is entirely mediated by one person.
In causal modeling, this is known as a bottleneck. Ricardo Silva and Richard Scheines pioneered the use of this intuition in New d-separation identification results for learning continuous latent variable models. They showed that if a single latent variable d-separates two sets of observed variables, it creates "vanishing tetrad differences"—a specific pattern where the product of two covariances equals the product of two others. This is the simplest form of a rank constraint: the covariance submatrix between those sets has a rank of 1.
From Graphs to Matrix Rank via Trek Separation
To move beyond simple single-mediator models, researchers needed a way to link complex graph topologies to matrix algebra. This link is provided by the Trek-Separation Theorem. A "trek" is a path between two variables that doesn't contain any "colliders" (arrows pointing at each other).
Luca Weihs, Elina Robeva, and colleagues demonstrated in Determinantal Generalizations of Instrumental Variables and Nested Covariance Determinants and Restricted Trek Separation in Gaussian Graphical Models that the rank of a covariance submatrix is exactly equal to the minimum number of nodes needed to "choke off" all treks between two sets of variables.
If you have two sets of variables, $X$ and $Y$, and the rank of their cross-covariance matrix is $k$, it implies there are exactly $k$ independent pathways (or latent variables) through which information flows from $X$ to $Y$. This allows researchers to "count" the number of hidden confounders without ever seeing them.
Identifying Hierarchies and Directionality
Recent work has extended these rank-based signatures to recover entire hierarchies of hidden variables. Biwei Huang and Feng Xie showed in Latent Hierarchical Causal Structure Discovery with Rank Constraints that by iteratively checking the rank of different submatrices, one can determine not just the number of latent variables, but how they are layered.
Rank constraints also help solve the "chicken-and-egg" problem of causal direction. In non-Gaussian systems, the rank of matrices constructed from higher-order statistics (like third or fourth-order cumulants) changes depending on which direction you assume the cause is flowing. Wei Chen and Kun Zhang leverage this in Identification of Causal Direction under an Arbitrary Number of Latent Confounders, using rank deficiency as a mathematical signature of the true causal flow.
Beyond Linear Models with Tensors
While early work focused on linear Gaussian models, the field has moved toward more general settings. When data is discrete or non-linear, simple covariance matrices are no longer enough. Instead, researchers look at the tensor rank of joint probability tables.
Ruichu Cai explains in Learning Discrete Latent Variable Structures with Tensor Rank Conditions that the algebraic rank of a contingency table is constrained by the state space size of the latent variable that d-separates the indicators. Similarly, Zhengming Chen showed in Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations that even non-linear dependencies can be mapped into discrete algebraic constraints using principled discretization, preserving the rank-based signals of hidden confounders.
Why This Works
The reason rank-based discovery is so robust is that it relies on structural constraints rather than specific functional forms. Whether a relationship is linear or non-linear, a bottleneck is still a bottleneck. By focusing on the dimensionality of the information flow (the rank) rather than the exact values of the correlations, these methods can:
- Detect Latents: Identify the presence of hidden variables that standard independence tests might miss (Xinshuai Dong and Biwei Huang in A Versatile Causal Discovery Framework to Allow Causally-Related Hidden Variables).
- Handle Selection Bias: Recover structures even when the data is systematically biased, because rank properties are often invariant to the distortions caused by selection (Haoyue Dai in Latent Variable Causal Discovery under Selection Bias).
- Automate Proxy Selection: Identify which observed variables can serve as reliable stand-ins for unmeasured causes (Feng Xie in Automating the Selection of Proxy Variables of Unmeasured Confounders).