Abstract
Given a response $Y$ and a vector $X = (X^1, \dots, X^d)$ of $d$ predictors, we investigate the problem of inferring direct causes of $Y$ among the vector $X$. Models for $Y$ that use all of its causal covariates as predictors enjoy the property of being invariant across different environments or interventional settings. Given data from such environments, this property has been exploited for causal discovery. Here, we extend this inference principle to situations in which some (discrete-valued) direct causes of $Y$ are unobserved. Such cases naturally give rise to switching regression models. We provide sufficient conditions for the existence, consistency and asymptotic normality of the MLE in linear switching regression models with Gaussian noise, and construct a test for the equality of such models. These results allow us to prove that the proposed causal discovery method obtains asymptotic false discovery control under mild conditions. We provide an algorithm, make available code, and test our method on simulated data. It is robust against model violations and outperforms state-of-the-art approaches. We further apply our method to a real data set, where we show that it does not only output causal predictors, but also a process-based clustering of data points, which could be of additional interest to practitioners.
Introduction
A fundamental challenge in scientific research is distinguishing between mere correlation and true causation. One of the most robust principles for identifying causal relationships is the Invariance Principle: if a model includes all direct causes of a target variable $Y$, the relationship between those causes and $Y$ should remain invariant across different environments, experimental conditions, or interventions. This principle forms the basis of Invariant Causal Prediction (ICP).
However, standard ICP faces a significant limitation: it assumes that all direct causes of $Y$ are observed. In many real-world systems, some influential factors are latent (unobserved). When a hidden variable influences $Y$, the relationship between the observed predictors and $Y$ appears to fluctuate across environments, leading standard ICP to incorrectly reject true causal predictors. This paper introduces ICPH (Invariant Causal Prediction with Hidden variables), a framework designed to maintain causal discovery power even when discrete latent variables act as unobserved "regime switches."
The Concept of h-Invariance
To address the problem of latent variables, the authors propose a relaxation of the standard invariance requirement called h-invariance (hidden invariance). Instead of requiring a single, stable linear relationship across all environments, the system is modeled as a Switching Regression Model.
In this setup, the relationship between predictors $X$ and the response $Y$ can exist in one of several discrete states or "regimes." Within each state, the relationship is linear and stable, but the specific parameters (slopes and intercepts) may differ between states. This effectively accounts for hidden categorical variables—such as different soil types in ecology or different market regimes in economics—that shift the underlying causal mechanism without being explicitly recorded in the dataset.
Theoretical Foundations and the ICPH Algorithm
The paper provides a rigorous statistical foundation for this approach, including:
- MLE Consistency: The authors prove the existence, consistency, and asymptotic normality of the Maximum Likelihood Estimator (MLE) for linear switching regression models under specific parameter constraints.
- Model Equality Testing: A novel statistical test is developed to determine if two switching regression models from different environments are identical. This is achieved by evaluating the joint overlap of Fisher confidence regions.
- The ICPH Algorithm: This algorithm searches through subsets of potential predictors. It only accepts a subset as "causal" if the switching regression models fitted to that subset are statistically indistinguishable across all observed environments.
Crucially, the authors prove that ICPH maintains asymptotic false discovery control. This means the probability of the algorithm falsely identifying a non-causal variable as a cause is bounded by a pre-defined significance level $\alpha$.
Experimental Results
Simulated Data
The method was tested against various Structural Causal Models (SCMs) designed to violate standard ICP assumptions. These simulations included non-Gaussian noise, heterogeneous variances, and varying numbers of latent states. ICPH consistently outperformed baseline methods, such as "k-means ICP" and constraint-based methods like JCI-PC. As sample sizes increased, ICPH demonstrated superior stability and a lower false discovery rate.
Real-World Application: Earth Science
The researchers applied ICPH to a dataset involving Sun-Induced Fluorescence (SIF), a proxy for photosynthetic activity in plants. The goal was to identify the causal drivers of SIF among predictors like shortwave radiation (SW), absorbed photosynthetically active radiation (APAR), and gross primary productivity (GPP).
In this context, the "hidden variable" is the vegetation type (e.g., cropland vs. evergreen forest), which is known to alter the SIF relationship. ICPH successfully:
- Identified APAR as the primary causal driver of SIF, aligning with biological consensus.
- Provided a process-based clustering of data points, correctly classifying over 95% of the data into their true (but unobserved) vegetation categories.
Conclusion and Future Work
ICPH extends the power of invariant causal discovery to more realistic scenarios where discrete latent factors are present. By treating these factors as regime switches, the framework provides a robust way to identify causal predictors while offering a useful byproduct: the ability to cluster data based on underlying physical or biological processes.
While highly effective for discrete latents, the method currently assumes a small number of states and linear relationships within those states. Future research may explore extensions into non-linear switching models and strategies to reduce the computational complexity associated with fitting non-convex switching regressions at scale.
Summary updated