Abstract
The Trek Separation Theorem (Sullivant et al. 2010) states necessary and sufficient conditions for a linear directed acyclic graphical model to entail for all possible values of its linear coefficients that the rank of various sub-matrices of the covariance matrix is less than or equal to n, for any given n. In this paper, I extend the Trek Separation Theorem in two ways: I prove that the same necessary and sufficient conditions apply even when the generating model is partially non-linear and contains some cycles. This justifies application of constraint-based causal search algorithms such as the BuildPureClusters algorithm (Silva et al. 2006) for discovering the causal structure of latent variable models to data generated by a wider class of causal models that may contain non-linear and cyclic relations among the latent variables.
Introduction
In fields such as psychology, sociology, and biology, researchers frequently investigate "latent variables"—hidden factors like intelligence or gene expression that cannot be observed directly. Instead, these factors are inferred through "indicators," such as test scores or protein levels. Traditionally, the mathematical tools used to discover these causal structures have relied on the strict assumptions that the underlying system is both linear and acyclic (lacking feedback loops).
However, real-world systems are rarely so simple. Feedback loops and non-linear interactions are common in biological and social systems. This research addresses a critical gap: determining whether the mathematical rules used to identify latent structures—specifically rank constraints in a covariance matrix—remain valid when these assumptions are relaxed.
Theoretical Framework: Extending Trek Separation
The Trek Separation Theorem provides the link between a graph's structure and specific rank constraints (such as vanishing tetrad differences) in the observed data. This paper introduces the concept of Linear Acyclicity (LA) below a choke set to extend this theorem.
The core of this approach is a structural division of the model:
- The Latent Layer: Relationships between hidden latent variables can be complex, non-linear, and cyclic.
- The Measurement Layer: The paths from latent variables to their measured indicators must remain linear and acyclic.
As shown in the model above, as long as the "measurement" portion of the model is linear, the rank constraints on the observed data are determined by the graph's structure, regardless of the complexity occurring "upstream" among the latent variables. The "choke set" (the latent parents of a group of indicators) acts as a bottleneck that preserves the mathematical signals used for causal discovery.
Experimental Validation
To validate this theory, simulations were performed using the TETRAD IV implementation of the BuildPureClusters algorithm. The study utilized synthetic datasets featuring 5 latent variables and 25 indicators, incorporating "impure" elements like indicators with multiple parents to test the algorithm's robustness. The researchers generated 100 datasets for various sample sizes (N=100, 500, 1000) and varying degrees of non-linearity, introduced via a cubic term with coefficients ranging from 0.0 to 0.05.
Key Results
The simulation results demonstrated that rank constraints are remarkably resilient to non-linearity and cycles in the latent structure. The BuildPureClusters algorithm maintained high performance across several metrics:
- Purity and Fraction: The algorithm successfully identified clusters where indicators shared a single latent parent (Purity) and identified the largest possible clusters (Fraction) even as non-linearity increased.
- Sample Size Sensitivity: While non-linearity made rank constraints slightly harder to detect at small sample sizes (N=100), at N=1000, the results were nearly identical to those of purely linear models.
- Robustness: The performance only degraded significantly when the relationship between the latent variable and its indicator became non-linear. If the non-linearity was confined to the relationships between the latents themselves, the algorithm remained highly effective.
Conclusion
This work provides the formal justification for applying existing constraint-based causal search algorithms to a much broader class of models. By proving that rank constraints hold in partially non-linear and cyclic systems, this research allows scientists to trust tools like BuildPureClusters even when the underlying real-world system involves complex feedback loops. Future work will explore how other functional forms of non-linearity, such as logarithmic or exponential relationships, affect the stability of these constraints.
Summary updated