Regime Identification for Improving Causal Analysis in Non-stationary Timeseries

Wasim Ahmad, Maha Shadaydeh, Joachim DenzlerarXiv 2024 · DOI · arXiv

Abstract

Time series data from real-world systems often display non-stationary behavior, indicating varying statistical characteristics over time. This inherent variability poses significant challenges in deciphering the underlying structural relationships within the data, particularly in correlation and causality analyses, model stability, etc. Recognizing distinct segments or regimes within multivariate time series data, characterized by relatively stable behavior and consistent statistical properties over extended periods, becomes crucial. In this study, we apply the regime identification (RegID) technique to multivariate time series, fundamentally designed to unveil locally stationary segments within data. The distinguishing features between regimes are identified using covariance matrices in a Riemannian space. We aim to highlight how regime identification contributes to improving the discovery of causal structures from multivariate non-stationary time series data. Our experiments, encompassing both synthetic and real-world datasets, highlight the effectiveness of regime-wise time series causal analysis. We validate our approach by first demonstrating improved causal structure discovery using synthetic data where the ground truth causal relationships are known. Subsequently, we apply this methodology to climate-ecosystem dataset, showcasing its applicability in real-world scenarios.

Introduction

Classical tools for causal inference typically rely on the assumption of stationarity—the idea that the statistical properties of a system, such as mean and variance, remain constant over time. However, real-world systems like global climate, financial markets, and biological processes are inherently non-stationary. They undergo "regime shifts" where the underlying causal dynamics change significantly. Analyzing such data as a single block often leads to "blurred" or incorrect causal conclusions because the shifting mechanisms are averaged out.

Illustration of a non-stationary multivariate time series with a clear regime change point (RCP) at index 65.

To address this, we propose a framework that segments non-stationary multivariate time series into locally stationary regimes before performing causal discovery.

Methodology: The RegID Framework

The proposed Regime Identification (RegID) technique follows a three-step process to partition data into stable segments:

  1. Windowing and Covariance Estimation: The multivariate time series is partitioned into small, overlapping time windows. For each window, a covariance matrix is computed to represent the local dependency structure between variables.
  2. Riemannian Geometry: Covariance matrices are Symmetric Positive Definite (SPD) matrices. Instead of treating them as points in a flat Euclidean space, we treat them as points on a curved Riemannian manifold. We employ the Log-Euclidean metric to calculate distances between these matrices, which better preserves the geometric properties of the data.
  3. Clustering: A $k$-means clustering algorithm is applied to the covariance matrices in the Riemannian space to group windows into distinct regimes.

Once regimes are identified, standard causal discovery algorithms (e.g., PCMCI+, VAR-GC) are applied to each regime independently, ensuring the stationarity assumption is met within each analysis block.

Experimental Results

The framework was validated using synthetic datasets with known ground-truth causal links and the FLUXNET2015 environmental dataset.

Performance and Stability

A key finding is the superiority of Riemannian distance over standard Euclidean distance. As shown in Figure 6, the Riemannian approach is significantly more stable across various window sizes (ranging from 15 to 90 data points), consistently identifying the correct regime transitions. In contrast, the Euclidean approach is highly sensitive to window size and often produces erratic results.

Comparison of regime identification performance between Euclidean and Riemannian distances across various window sizes, demonstrating the superior stability of the Riemannian approach.

On synthetic data, integrating RegID with causal discovery led to substantial performance gains. For instance, the F-score for the CDMI algorithm improved from 0.72 to 0.91 when RegID was utilized. Similar improvements were observed across other methods like PCMCI+.

Real-world Application: Climate-Ecosystem Dynamics

In the FLUXNET data, RegID successfully identified regimes corresponding to seasonal cycles. Causal analysis within these regimes revealed bidirectional links between climate variables (temperature, radiation) and ecosystem responses (productivity, respiration) in forest sites, while identifying more unidirectional links in grassland ecosystems.

Conclusion and Future Work

RegID effectively addresses the non-stationarity problem by creating stationary environments for existing causal algorithms. While the Riemannian approach is robust, challenges remain in handling very high-dimensional data where covariance matrices become large. Future work will explore dimensionality reduction and automated parameter selection to further enhance the framework's applicability to complex real-world datasets.

Summary updated

Abstract

Time series data from real-world systems often display non-stationary behavior, indicating varying statistical characteristics over time. This inherent variability poses significant challenges in deciphering the underlying structural relationships within the data, particularly in correlation and causality analyses, model stability, etc. Recognizing distinct segments or regimes within multivariate time series data, characterized by relatively stable behavior and consistent statistical properties over extended periods, becomes crucial. In this study, we apply the regime identification (RegID) technique to multivariate time series, fundamentally designed to unveil locally stationary segments within data. The distinguishing features between regimes are identified using covariance matrices in a Riemannian space. We aim to highlight how regime identification contributes to improving the discovery of causal structures from multivariate non-stationary time series data. Our experiments, encompassing both synthetic and real-world datasets, highlight the effectiveness of regime-wise time series causal analysis. We validate our approach by first demonstrating improved causal structure discovery using synthetic data where the ground truth causal relationships are known. Subsequently, we apply this methodology to climate-ecosystem dataset, showcasing its applicability in real-world scenarios.
BibTeX
@misc{ahmad2024regime,
  author = {Wasim Ahmad and Maha Shadaydeh and Joachim Denzler},
  title = {{Regime Identification for Improving Causal Analysis in Non-stationary Timeseries}},
  year = {2024},
  eprint = {2405.02315},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2405.02315},
}
Figure