Star-causality and factor analysis: old stories and new perspectives

Lei XuApplied Informatics 2017 · DOI

Advances in causal discovery from data have become a cornerstone of modern machine learning. This paper, published in Applied Informatics, bridges the gap between classical statistical techniques and modern discovery by revisiting the relationship between factor analysis and causal discovery. Author Lei Xu provides a fresh perspective on how we can identify hidden "common causes" that drive observed correlations in data, moving from rigid statistical tests toward flexible, optimization-based frameworks.

The Challenge: Inferring Latent Causes

In data science, the Common Cause Principle suggests that if two variables are correlated, there is likely a causal explanation: either one causes the other, or a third, unobserved variable (a latent factor) causes both. The challenge lies in "causal discovery"—automatically inferring these structures from raw data.

Traditional methods, like those pioneered by Judea Pearl, often rely on strict conditions (such as the TETRAD equations) that are rarely met perfectly in real-world, noisy datasets. Furthermore, these methods often struggle to scale or handle complex hierarchies where groups of variables are influenced by multiple hidden factors. Xu addresses this by asking how these classical causal models can be made more robust and generalizable to modern, high-dimensional datasets.

Structural Models: Stars and Trees

The paper focuses on two primary structures that serve as the building blocks of causal networks:

  1. Star-decomposable models: A central latent cause $W$ influences multiple observed effects $X_i$.
  2. Tree-decomposable models: Hierarchies of latent causes that form more complex branching structures.
Figure 1: Visualizing star-decomposable (a), triplet star (b), tree-decomposable (c) models, and their relationship to factor analysis (d).

As shown in Figure 1, the "star" structure is the fundamental unit. By mastering the discovery of "stars," researchers can build complex "trees" by linking latent nodes together. The paper formally demonstrates that a star-causal model is mathematically equivalent to a one-factor analysis model, allowing researchers to use the well-established machinery of factor analysis to solve causal problems.

Methodological Innovations

Xu proposes three key methodological shifts to modernize causal discovery:

1. From Testing to Optimization

Instead of simply checking if a statistical condition (like the TETRAD equality) is exactly zero, the paper suggests a constrained optimization approach. By minimizing the "difference" from these conditions while applying a sparse learning penalty, the model is "forced" to find the most likely, simplest causal structure. This makes the discovery process far more robust to noise.

2. Block Star-Causality

Xu generalizes the model from single variables to "blocks" of variables. In this view, a vector of latent factors influences groups of observed variables. This "block" approach is more suitable for high-dimensional data where variables naturally cluster together, allowing for a top-down or bottom-up approach to discovery.

3. The Power of Second-Order Statistics

For Gaussian variables, the paper emphasizes that the entire causal structure can be recovered using only second-order statistics (correlation coefficients). This is significantly more computationally efficient than methods requiring higher-order statistics, making it viable for large-scale data analysis.

Theoretical Roadmap and Future Directions

The paper provides a theoretical roadmap rather than a benchmark table. Its main contributions include a unified framework linking Reichenbach’s common cause principle, Pearl’s causal trees, and Spearman’s factor analysis. It derives specific objective functions for learning approximate star and block-star structures and identifies the necessary conditions for when a covariance matrix can be uniquely decomposed into a causal factor model.

While the current mathematical framework assumes Gaussian distributions, the author suggests a two-stage approach for non-Gaussian data. Future work involves applying these "block" methods to linear causal models and exploring the trade-offs between top-down and bottom-up discovery in massive, hierarchical datasets.

Summary updated

Abstract

Advances in causal discovery from data are becoming a widespread topic in machine learning these recent years. In this paper, studies on conditional independence-based causality are briefly reviewed along a line of observable two-variable, three-variable, star decomposable, and tree decomposable, as well as their relationship to factor analysis. Then, developments along this line are further addressed from three perspectives with a number of issues, especially on learning approximate star decomposable, and tree decomposable, as well as their generalisations to block star-causality analysis on factor analysis and block tree decomposable analysis on linear causal model.
BibTeX
@inproceedings{xu2017star,
  author = {Lei Xu},
  title = {{Star-causality and factor analysis: old stories and new perspectives}},
  booktitle = {Applied Informatics},
  year = {2017},
  doi = {10.1186/s40535-017-0046-1},
}
Figure