Causal Discovery from Sparse Time-Series Data Using Echo State Network

Haonan Chen, Bo Yuan Chang, Mohamed A. Naiel, Georges Younes, Steven Wardell, Stan Kleinikkink, John S. ZelekarXiv 2022 · arXiv

Abstract

Understanding causal relationships in time-series data is critical for system diagnostics, yet real-world data often suffers from sparsity due to missing entries. This paper presents a novel two-stage pipeline that integrates Gaussian Process Regression (GPR) for data imputation with an Echo State Network (ESN) for Granger Causality estimation. Evaluated on the Tennessee Eastman Process dataset with 10% missing data, the proposed GC-ESN system demonstrates superior robustness and accuracy compared to traditional linear and structural models, maintaining performance levels close to those achieved with complete datasets.

Introduction

In complex industrial and biological systems, identifying the directional influence between variables—causal discovery—is essential for fault detection and prevention. However, sensor data is frequently "sparse" or non-uniformly sampled due to hardware failures, network latency, or human error. Traditional causal discovery algorithms, such as Granger Causality or Structural Expectation Maximization (SEM), often fail when confronted with gaps in time-series data. This research addresses the challenge of performing reliable causal inference from imperfect, sparse datasets without requiring expensive hardware upgrades.

Methodology

The proposed framework utilizes a modular two-stage approach to bridge the gap between sparse observations and causal modeling.

The proposed two-stage pipeline: GPR for data imputation followed by GC-ESN for causal estimation.
  1. Data Imputation via Gaussian Process Regression (GPR): Instead of simple linear interpolation, GPR is employed to model the underlying distribution of the sparse data. By providing a probabilistic estimation of missing values, GPR effectively reconstructs the continuous signal from fragmented observations.
  2. Causal Estimation via Echo State Network (ESN): The reconstructed data is processed by a Granger Causality - Echo State Network (GC-ESN). As a form of reservoir computing, the ESN maps inputs into a high-dimensional nonlinear reservoir. This architecture is particularly adept at capturing the chaotic, nonlinear dynamics inherent in industrial processes, allowing for the detection of subtle causal signatures that linear models might overlook.

Experimental Setup

The system was validated using the Tennessee Eastman (TE) Process Dataset, a benchmark in chemical engineering simulating a multi-unit plant. The study focused on 8 key sensor readings (e.g., pressure, temperature, flow rates) over 200 operation cycles. To simulate real-world conditions, the authors "contaminated" the dataset by randomly removing 10% of the entries. The performance of the GPR+ESN pipeline was compared against three baselines: Structural Expectation Maximization (SEM), Sub-sampled Linear Auto-Regression Absolute Coefficients (SLARAC), and Multivariate Granger Causality (MVGC).

Results and Discussion

The experimental results highlight the robustness of the GC-ESN approach. Performance was measured using the Matthews Correlation Coefficient (MCC) and Area Under the ROC Curve (AUC).

Receiver Operating Characteristic (ROC) curves showing the performance of GC-ESN versus baseline methods on sparse (10% missing) data.

The GPR + GC-ESN system achieved an MCC of 0.26 on the 10% missing data, remarkably close to its performance on the complete dataset (MCC 0.29). In contrast, baseline methods showed significant degradation:

  • GPR + SLARAC: 0.15 MCC
  • GPR + MVGC: 0.14 MCC
  • SEM: -0.18 MCC (performing worse than random guessing)

As shown in the ROC curves, the proposed system achieved an AUC of 0.662, significantly outperforming SLARAC (0.412) and MVGC (0.449). These results suggest that the ESN's nonlinear reservoir is more resilient to the "lossy" nature of GPR-interpolated data than traditional linear auto-regressive models.

Conclusion

This research demonstrates that combining probabilistic data filling with reservoir computing provides a stable and effective solution for causal discovery in sparse time-series. While the system outperforms existing automated methods, it does not yet match the precision of a human subject-matter expert. Future work will explore the limits of this approach under higher percentages of data sparsity and investigate architectural refinements to further distinguish true causation from noise in high-dimensional environments.

Summary updated

Abstract

Causal discovery between collections of time-series data can help diagnose causes of symptoms and hopefully prevent faults before they occur. However, reliable causal discovery can be very challenging, especially when the data acquisition rate varies (i.e., non-uniform data sampling), or in the presence of missing data points (e.g., sparse data sampling). To address these issues, we proposed a new system comprised of two parts, the first part fills missing data with a Gaussian Process Regression, and the second part leverages an Echo State Network, which is a type of reservoir computer (i.e., used for chaotic system modelling) for Causal discovery. We evaluate the performance of our proposed system against three other off-the-shelf causal discovery algorithms, namely, structural expectation-maximization, sub-sampled linear auto-regression absolute coefficients, and multivariate Granger Causality with vector auto-regressive using the Tennessee Eastman chemical dataset; we report on their corresponding Matthews Correlation Coefficient(MCC) and Receiver Operating Characteristic curves (ROC) and show that the proposed system outperforms existing algorithms, demonstrating the viability of our approach to discover causal relationships in a complex system with missing entries.
BibTeX
@misc{chen2022causal,
  author = {Haonan Chen and Bo Yuan Chang and Mohamed A. Naiel and Georges Younes and Steven Wardell and Stan Kleinikkink and John S. Zelek},
  title = {{Causal Discovery from Sparse Time-Series Data Using Echo State Network}},
  year = {2022},
  eprint = {2201.02933},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2201.02933},
}
Figure