Abstract
We propose a new method of discovering causal relationships in temporal data based on the notion of causal compression. To this end, we adopt the Pearlian graph setting and the directed information as an information theoretic tool for quantifying causality. We introduce chain rule for directed information and use it to motivate causal sparsity. We show two applications of the proposed method: causal time series segmentation which selects time points capturing the incoming and outgoing causal flow between time points belonging to different signals, and causal bipartite graph recovery. We prove that modelling of causality in the adopted set-up only requires estimating the copula density of the data distribution and thus does not depend on its marginals. We evaluate the method on time resolved gene expression data.
Introduction: The "Shortcut" to Causal Discovery
In complex fields like genomics or finance, researchers often deal with multiple time series and aim to understand the underlying influences between them. Traditional causal discovery methods typically attempt to reconstruct the entire network of interactions. However, following the principle of Vladimir Vapnik—"avoid solving a more general problem as an intermediate step"—the authors of this paper argue that full network reconstruction is often unnecessary overkill.
If the goal is to understand the relationship between two specific signals, can we identify specific causal flows (incoming, outgoing, or instantaneous) without mapping the whole system? Causal Compression provides a "shortcut" to identify these relationships efficiently and robustly.
Methodology
The framework combines information theory with sparse optimization through two primary pillars:
- Directed Information (DI): Unlike standard correlation, DI is asymmetrical and measures the flow of information from variable $X$ to variable $Y$ specifically over time.
- Sparse Optimization: Causal discovery is treated as a compression problem. The algorithm searches for the smallest subset of time points (a sparse representation) that preserves the maximum amount of directed information.
To implement this, the authors developed a "Chain Rule" for directed information, proving that enforcing sparsity naturally selects true causal parents. The optimization is handled by a greedy Stagewise Forward Algorithm, which iteratively adds time points based on their causal contribution.
The Copula Theorem
A significant theoretical contribution is the Copula Theorem. The authors prove that causal relationships in this framework depend solely on the dependence structure (the copula) of the data, rather than the individual marginal distributions. This allows the method to remain effective on "meta-Gaussian" data, making it far more flexible than standard linear models that assume Gaussianity.
Empirical Evaluation
Synthetic Data
The method was tested on a 6th-order Markov model with 500 samples. The algorithm achieved perfect recovery, identifying the exact ground-truth bipartite graph and all causal segments (3 outgoing, 4 incoming, and 2 instantaneous links).
Biological Application: HCV Gene Expression
The researchers applied Causal Compression to human Hepatitis C Virus (HCV) patient data, tracking gene expressions (e.g., STAT1 and IFIT3) during antiviral treatment.
- Marked Responders: In patients who recovered, the treatment appeared to decouple the normally tight instantaneous relationship between STAT1 and IFIT3.
- Predictive Influence: The model revealed that in successful responders, the initial levels of IFIT3 had a significantly higher causal influence on later STAT1 values compared to poor responders, suggesting a potential biomarker for treatment efficacy.
Conclusion and Future Work
Causal Compression offers a computationally lean and targeted alternative to full network reconstruction. By focusing on specific causal functions rather than the entire graph, it provides clearer insights into complex systems like gene regulation.
Future work aims to address current limitations, such as the requirement for a known global temporal ordering and the reliance on linear noisy projections within the optimization step. Scaling the bipartite approach to handle thousands of simultaneous time series remains a key area for further development.
Summary updated