Explainable Tensorized Neural Ordinary Differential Equations forArbitrary-step Time Series Prediction

Penglei Gao, Xi Yang, Rui Zhang, Kaizhu Huang

Introduction

Time series forecasting is a cornerstone of data science, yet traditional models often struggle with two major hurdles: the rigidity of discrete-time predictions and the lack of interpretability in multivariate settings. Standard Recurrent Neural Networks (RNNs) are typically constrained to fixed intervals, making them ill-suited for real-world scenarios requiring predictions at arbitrary future time points. Furthermore, as the number of input variables increases, these models often become "black boxes," obscuring the specific drivers of a forecast.

Comparison of discrete multi-step prediction versus continuous arbitrary-step prediction.

To address these challenges, we present ETN-ODE (Explainable Tensorized Neural Ordinary Differential Equations), a novel framework that leverages continuous-time modeling and dual-attention mechanisms to provide accurate, explainable, and flexible time series forecasts.

Methodology: The ETN-ODE Framework

The ETN-ODE architecture is built upon three integrated components designed to handle complex multivariate dynamics while maintaining transparency.

High-level architecture of the ETN-ODE framework, integrating TGRU, Tandem Attention, and Neural ODEs.

1. Tensorized GRU (TGRU)

Unlike standard Gated Recurrent Units (GRUs) that represent hidden states as vectors, the TGRU utilizes a matrix (2D tensor). This structural change allows the model to maintain independent dynamics for each input feature, preventing the premature "smearing" of information across variables. This approach is not only more representative of multivariate systems but also more parameter-efficient than traditional tensorized LSTM variants.

2. Tandem Attention Mechanism

Explainability is achieved through a dual-layered attention system. The temporal attention layer identifies critical historical moments for each variable, while the variable-wise attention layer quantifies the overall contribution of each exogenous variable to the final prediction. This "tandem" approach provides a clear map of the model's decision-making process.

3. Neural ODE Network

The core of ETN-ODE's continuous-time capability is the Neural Ordinary Differential Equation (ODE) layer. Instead of mapping inputs directly to future states, the model uses the context vector from the attention layers to parameterize the derivative (rate of change) of the latent state. An ODE solver then integrates this derivative, allowing the model to evolve the state forward to any arbitrary time point $T+m$, regardless of the sampling frequency of the training data.

Experimental Evaluation

The model was rigorously tested against state-of-the-art baselines (e.g., Latent-ODE, TLSTM, IMV-tensor) across four real-world datasets: SML2010 (indoor temperature), PM2.5 (air quality), Energy (appliance consumption), and NASDAQ100 (financial stock prices).

Key Results

  • Superior Accuracy: ETN-ODE consistently achieved the lowest Root Mean Square Error (RMSE) across all datasets. Notably, on the SML2010 dataset, it outperformed the strong IMV-tensor baseline by an average of 17.79%.
  • Arbitrary-Step Flexibility: The model demonstrated a significant advantage in predicting values at non-integer time steps, capturing periodic trends that discrete models missed.
  • Interpretability: Visualization of the tandem attention weights confirmed that the model identifies physically meaningful relationships, such as the impact of outdoor temperature and CO2 levels on indoor climate.

Conclusion

ETN-ODE represents a significant step forward in time series forecasting by bridging the gap between continuous-time modeling and explainable AI. By moving from discrete vectors to tensorized states and parameterizing derivatives rather than direct transitions, the framework offers unparalleled flexibility and insight. Future work will focus on optimizing computational efficiency for edge deployment and extending the model to handle irregularly sampled input data.

Summary updated

BibTeX
@article{gao2023explainable,
  author = {Penglei Gao and Xi Yang and Rui Zhang and Kaizhu Huang},
  title = {{Explainable Tensorized Neural Ordinary Differential Equations forArbitrary-step Time Series Prediction}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2023},
  doi = {10.1109/tkde.2022.3167536},
}
Figure