TL;DR
This paper introduces a novel causality detection criterion based on conditional relative entropy, which is formally linked to conditional mutual information. This new approach, when applied to human gait data, demonstrates superior ability to detect non-linear causal relationships compared to traditional methods like Granger causality.
What Problem Does This Paper Address?
The fundamental question in time series analysis is whether one series acts as an input or driver for another. Traditional methods, such as those by Sims, Geweke-Meese-Dent, and Granger, primarily rely on modeling time series with vector autoregressive models, which are effective for linear relationships. However, real-world systems, including human movement, often exhibit complex non-linear dynamics. The challenge is to develop a causality detection method that can reliably identify these non-linear causal influences, which are missed by linear models.
Methods
The core of this paper is the introduction of Conditional Relative Entropy (CRE) as a causality criterion. The authors derive a theorem that formally connects CRE to Conditional Mutual Information (CMI). This connection is crucial because it allows the estimation of CRE using existing, well-developed estimators for entropy and mutual information.
The paper then details how CRE can be computed by leveraging these estimators. Specifically, it shows that CRE is equivalent to CMI. The authors further express CMI in terms of unconditional entropies, which are then estimated using a k-nearest neighbor approach. For modeling the time-varying dynamics of human gait, they employ a VARMA-GARCH (Vector Autoregressive Moving Average - Generalized Autoregressive Conditional Heteroscedasticity) model. This model captures both the time-varying mean (oscillating behavior) and the time-varying variance of the gait data.
Datasets
The study utilizes real-world time series data of human gait. This data was collected using a Tekscan FScan system, sampling forces exerted on the soles of the feet at 100 Hz. The experiments involved data from 7 patients with Parkinson's disease, resulting in 20 experimental time series. The paper does not introduce a new benchmark dataset but uses this existing real-world data for validation.
Key Insights and Ideas
- Formalizing Causality with Information Theory: The paper provides a rigorous information-theoretic foundation for causality detection by linking conditional relative entropy to conditional mutual information. This moves beyond heuristic applications of information theory.
- Bridging Theory and Practice: By showing that CRE can be estimated using readily available mutual information and entropy estimators, the paper makes a powerful theoretical concept practically applicable.
- Detecting Non-Linear Causality: The most significant insight is that CMI, derived from the CRE framework, is sensitive to non-linear causal relationships, a capability lacking in traditional linear methods.
- VARMA-GARCH for Gait: The VARMA-GARCH model is shown to be a suitable tool for capturing the complex, time-varying dynamics (both mean and variance) inherent in human gait data.
Key Results
- Linear Regime Performance: In the linear regime, the proposed Conditional Mutual Information (CMI) criterion performed comparably to established methods like Sims, Geweke-Meese-Dent, and Granger causality, correctly identifying causal dependencies in 100% of the tested cases (left to right) and 95% (right to left).
- Non-Linear Regime Superiority: When applied to residual time series (after removing linear components) and analyzed within the VARMA-GARCH framework, CMI significantly outperformed the traditional methods. While Sims, Geweke-Meese-Dent, and Granger detected 0% of non-linear causal relationships, CMI detected 58.82% (left to right) and 66.67% (right to left) of these non-linear dependencies.
- Model Validation: The VARMA-GARCH model effectively captured the time-varying variance in gait residuals, indicating its suitability for modeling such complex biological signals.
Limitations and Future Work
- Second-Order Dependencies: The paper notes that the GARCH model, while useful, primarily detects second-order dependencies. The authors propose a performance measure to quantify the correspondence between GARCH and CMI, acknowledging that CMI might detect relationships beyond what GARCH can capture.
- Parameter Sensitivity: While the paper states that the choice of parameters ($n_1$, $n_2$) for CMI was not critical in their experiments, it's a common area for future investigation to explore the robustness of the method to parameter choices.
- Future Directions: The paper implicitly suggests that further exploration of non-linear causality in biological and other complex systems using this information-theoretic framework is warranted. The application of CMI to other domains beyond human gait is a natural next step.
Summary updated