Peer Review #2 of "CGCNImp: a causal graph convolutional network for multivariate time series imputation (v0.1)"

Caizheng Liu, Guangfan Cui, Shenghua Liu, M Amiri, R Jensen, L Anghinoni, D Vega-Oliveros, T Silva, L Zhao, M Azur, E Stuart, C Frangakis, P Leaf, F Bashir, H.-l Wei, S Bauer, B Schlkopf, J Peters, S Bengio, O Vinyals, N Jaitly, N Shazeer, M Berglund, T Raiko, M Honkala, L Krkkinen, A Vetek, J Karhunen, L Boz, T Csrg, W Heim, S Buuren, C Groothuis-Oudshoorn, C Li, W Su, T Mckenzie, F Hsu, S Lin, J Hsu, P Gibbons, W Cao, D Wang, J Li, H Zhou, L Li, Y Li, Z Che, S Purushotham, K Cho, D Sontag, Y Liu, J Cheema, M Clairbaux, J Fort, P Mathewson, W Porter, H Strm, D Grmillet, S, TOpenalex Unknown 2022 · DOI · OpenAlex

Introduction

Multivariate time series (MTS) data is foundational to modern analytics, from monitoring urban air quality to tracking patient health in intensive care units. However, real-world data collection is frequently plagued by missing values due to sensor malfunctions, transmission failures, or human error. Traditional imputation methods often fail to account for the intricate web of causal relationships and temporal dynamics inherent in these datasets.

A recent study introduces CGCNImp, a novel deep learning framework designed to address these challenges. By integrating Neural Granger Causality with Graph Convolutional Networks (GCNs), CGCNImp explicitly models the "who-affects-whom" structure of data to reconstruct missing values with high fidelity.

The Challenge of Missing Data

Imputing missing values in MTS is difficult because of two primary factors:

  1. Attribute Correlation: Variables are often physically or logically linked. For instance, wind speed at one location directly influences pollutant concentrations at another.
  2. Temporal Dependency: Time series data is non-stationary. While past values inform the present, their relevance decays over time—a phenomenon known as temporal decay.

Existing solutions like K-Nearest Neighbors (KNN) or standard Long Short-Term Memory (LSTM) networks often overlook directed causal relationships or struggle with the noise inherent in sensor data.

The CGCNImp Architecture

The CGCNImp model operates through four specialized components:

1. Causal Discovery Module

Instead of assuming static relationships, the model employs Neural Granger Causality to learn a directed adjacency matrix. This causal graph identifies if the history of one variable helps predict the future of another, providing a more physically accurate representation of the system than simple correlation.

Learned causal graph for the KDD CUP 2018 dataset, showing the complex inter-variable dependencies captured by the model.

2. Correlation Dependency Module

Utilizing the learned causal graph, a Graph Convolutional Network (GCN) processes the data. This allows the model to propagate information across related attributes based on their causal structure, effectively "smoothing" the imputation process.

3. Temporal Dependency Module

This module uses an attention-driven LSTM combined with a time lag matrix. The time lag matrix calculates a decay vector ($\alpha$) that reduces the influence of older observations as the duration of a data gap increases, acknowledging that the system's state becomes more uncertain over time.

4. Total Variation (TV) Reconstruction

To mitigate sensor noise, the model incorporates TV regularization, encouraging imputed values to remain smooth and consistent with neighboring data points.

Experimental Evaluation

The researchers tested CGCNImp on two diverse real-world datasets:

  • KDD CUP 2018: A meteorological dataset from Beijing containing 121 variables, including air quality and weather metrics.
  • Bird Migration: Trace data for two species (Anser albifrons and Anser fabalis), featuring attributes such as longitude, latitude, speed, and heading.
Learned causal graph for the Bird Migration dataset, illustrating directed relationships between movement attributes.

Results and Discussion

CGCNImp consistently outperformed eight baseline methods, including BRITS, GRUI, and E2E-GAN, across various missing rate scenarios.

  • High Accuracy: On the KDD dataset with a 10% missing rate, CGCNImp achieved a Mean Absolute Error (MAE) of 0.062, outperforming the next best model, E2E-GAN (0.074).
  • Robustness: The model maintained strong performance even at a 90% missing rate. For the Bird Migration dataset, it achieved an MAE of 0.2014, while traditional methods like BRITS saw errors rise to 0.2896.
  • Ablation Studies: Removing the causal graph or the noise reduction module significantly increased error rates, confirming that both spatial causal modeling and temporal smoothing are essential for high-fidelity imputation.

Conclusion

CGCNImp demonstrates that understanding the causal forces driving a system is key to accurately reconstructing its missing parts. By moving beyond simple correlation to directed causal graphs, this framework provides a robust solution for handling high-dimensional, "noisy" time series data in critical real-world applications.

Summary updated

Abstract

Background: Multivariate time series data generally contains missing values, which can be an obstacle to subsequent analysis and compromise downstream applications.One challenge in this endeavor is presence of the missing values brought about by sensor failure and transmission packet loss.Imputation is the usual remedy in such circumstances.However, in some multivariate time series data, the complex correlation and temporal dependencies, coupled with the non-stationarity of the data, make imputation difficult. Mehods:To address this problem, we propose a novel model for multivariate time series imputation called CGCNImp that considers both correlation and temporal dependency modeling.The correlation dependency module leverages neural Granger causality and a GCN to capture the correlation dependencies among different attributes of the time series data, while the temporal dependency module relies on an attention-driven LSTM and a time lag matrix to learn its dependencies.Missing values and noise are addressed with total variation reconstruction. Results:We conduct thorough empirical analyses on two real-world datasets.Imputation results show that CGCNImp achieves state-of-the-art performance when compared to previous methods.
BibTeX
@inproceedings{liu2022peer,
  author = {Caizheng Liu and Guangfan Cui and Shenghua Liu and M Amiri and R Jensen and L Anghinoni and D Vega-Oliveros and T Silva and L Zhao and M Azur and E Stuart and C Frangakis and P Leaf and F Bashir and H.-L Wei and S Bauer and B Schlkopf and J Peters and S Bengio and O Vinyals and N Jaitly and N Shazeer and M Berglund and T Raiko and M Honkala and L Krkkinen and A Vetek and J Karhunen and L Boz and T Csrg and W Heim and S Buuren and C Groothuis-Oudshoorn and C Li and W Su and T Mckenzie and F Hsu and S Lin and J Hsu and P Gibbons and W Cao and D Wang and J Li and H Zhou and L Li and Y Li and Z Che and S Purushotham and K Cho and D Sontag and Y Liu and J Cheema and M Clairbaux and J Fort and P Mathewson and W Porter and H Strm and D Grmillet and G Cui and Y Liu and S and T},
  title = {{Peer Review \#2 of "CGCNImp: a causal graph convolutional network for multivariate time series imputation (v0.1)"}},
  booktitle = {Openalex Unknown},
  year = {2022},
  doi = {10.7287/peerj-cs.966v0.1/reviews/2},
}
Figure