Abstract
Causal Structure Learning (CSL) is a powerful tool for uncovering cause-and-effect relationships within complex datasets. However, traditional CSL algorithms often struggle with stability and interpretability, particularly in high-noise environments. This paper introduces RPC (Robust pc.stable), a novel algorithm featuring a self-correction mechanism that integrates domain knowledge and consistency checks. By addressing error propagation caused by misleading statistical tests, RPC achieves superior accuracy on synthetic benchmarks and provides meaningful insights into real-world phenomena, such as the shift in global investment preferences during the COVID-19 pandemic.
Introduction: The Challenge of Noise in Causal Discovery
Discovering Directed Acyclic Graphs (DAGs) from observational data is the primary goal of CSL. Most constraint-based algorithms rely on Conditional Independence (CI) tests to determine the presence and direction of edges between variables. In ideal conditions, these tests accurately map the underlying causal structure. However, real-world data is frequently "noisy," leading to misleading p-values.
A single incorrect CI test can trigger a cascade of errors, known as error propagation. For instance, a false independence result can lead to "self-conflicting" structures, such as cycles or bi-directed edges (where arrows point in both directions between two nodes), which are logically impossible in a DAG. Furthermore, standard algorithms often ignore existing domain knowledge—established truths that could otherwise constrain the search space and prevent nonsensical results.
Methodology: The RPC* Framework
The proposed Robust pc.stable (RPC*) algorithm addresses these limitations through three core innovations:
- Compound Scoring: Rather than relying on a single p-value, RPC* employs a compound score derived from multiple statistical calculations. This provides a robust buffer against outliers and reduces the likelihood of a single noisy data point derailing the entire structure.
- Self-Correction and Consistency Testing: The algorithm incorporates a diagnostic phase that checks the output for logical consistency. If structural contradictions like bi-oriented edges or cycles are detected, RPC* triggers a "Separation Set Adjusting" phase. This iterative process re-evaluates suspicious links until a valid, consistent DAG is formed.
- Domain Knowledge (DK) Embedding: RPC* allows for the integration of prior knowledge. In the context of the COVID-19 study, for example, the algorithm was constrained to prevent financial market fluctuations from being identified as causes of the initial viral outbreak, ensuring the output remained grounded in reality.
Experimental Evaluation and Results
Synthetic Benchmarks
The researchers evaluated RPC against several classic Bayesian networks (e.g., Asia, Sachs, Child) under varying noise conditions. RPC consistently outperformed ten influential baseline algorithms, achieving the lowest Minimal Average Structural Hamming Distance (MASHD). This indicates that the graphs learned by RPC* were the most faithful to the ground truth.
Real-World Case Study: COVID-19 and Investment Shifts
The algorithm was applied to financial data spanning the onset of the COVID-19 pandemic, including stock indices (DJI, S&P 500, NASDAQ), treasury yields, and commodities like Gold and Oil.
As shown in the comparative analysis, traditional algorithms like pc.stable produced a high number of bi-directional edges, rendering the results uninterpretable. In contrast, RPC* (and to a lesser extent, the hybrid MMHC method) produced stable, consistent structures.
The causal chains discovered by RPC revealed a significant shift in human investment preference. Before the pandemic, the 10-year treasury yield acted as the primary market "anchor." Following the outbreak, this role shifted to Gold*, confirming the "flight to safety" phenomenon where investors pivot toward traditional safe-haven assets during periods of extreme global uncertainty. These findings aligned closely with expert financial analysis and public sentiment data.
Conclusion
RPC* represents a significant step forward in making Causal Structure Learning more robust and interpretable. By treating structural contradictions as signals for correction rather than final outputs, the algorithm effectively mitigates the impact of noise. While future work will address latent variables and computational efficiency, the current framework demonstrates that prioritizing stability and domain integration can yield profound insights into complex human systems.
Summary updated