EgPDE-Net: Building Continuous Neural Networks for Time Series Prediction with Exogenous Variables

Penglei Gao, Xi Yang, Kaizhu Huang, Rui Zhang, Ping Guo, John Yannis Goulermas

Abstract

Predicting multivariate time series at arbitrary time points remains a challenge for traditional discrete-time models. We present EgPDE-Net, a novel continuous-time neural network that frames time series forecasting as a Partial Differential Equation (PDE) problem. By reducing the PDE to a tractable, exogenous-guided Ordinary Differential Equation (ODE), EgPDE-Net explicitly models the influence of external factors on target variables. Experimental results across four diverse datasets demonstrate that EgPDE-Net outperforms state-of-the-art continuous models, reducing RMSE by approximately 10% and MAE by 14% for arbitrary-step predictions.

Introduction

Most deep learning architectures for time series, such as Recurrent Neural Networks (RNNs), are inherently discrete. They are optimized for fixed-interval forecasting (e.g., $T+1, T+2$). However, many real-world applications require "arbitrary-step" prediction, where forecasts are needed at irregular or continuous time points.

While Neural Ordinary Differential Equations (Neural ODEs) provide a framework for continuous-time modeling, they often struggle with multivariate datasets where exogenous variables (external drivers like weather or power load) significantly impact the target series. Existing models frequently fail to capture the complex, time-dependent correlations between these external factors and the system's dynamics.

Comparison between discrete multi-step prediction and the proposed arbitrary-step prediction using PDE components.

Methodology: The EgPDE-Net Architecture

The core innovation of EgPDE-Net is treating the dynamical system as a complex PDE. This approach is more robust than standard ODEs for modeling systems influenced by multiple independent variables. To ensure mathematical tractability, the authors utilize a lemma to reduce the PDE into a regularized, exogenous-guided ODE.

The architecture is divided into two primary pipelines:

  1. Exogenous Pipeline: This branch processes external variables through a self-attention block to capture both global and local correlations. An ODE solver then generates "partial differential weights," representing the time-varying influence of these external factors.
  2. Target Pipeline: The target series is encoded using a Gated Recurrent Unit (GRU). A second ODE solver uses the weights from the exogenous pipeline to guide the trajectory of the hidden states, enabling continuous prediction at any point $t$.
The dual-pipeline architecture of EgPDE-Net, featuring exogenous and target processing branches guided by ODE solvers.

By parameterizing the derivative of the hidden state with these two components, the model effectively "guides" the forecast based on environmental changes.

Experimental Evaluation

The model was evaluated on four multivariate datasets: SML2010 (indoor temperature), Electricity (homestead consumption), and ETTh1/ETTh2 (transformer temperature). To test arbitrary-step prediction, the datasets were re-sampled so the model trained on integral steps but was tested on mid-step points (e.g., $T+1.5$).

Key Results

  • Superior Accuracy: EgPDE-Net reduced RMSE by 9.85% and MAE by 13.98% compared to the best continuous baseline, ETN-ODE.
  • Capturing Dynamics: In periodic datasets like SML2010, the model was notably better at capturing peaks and troughs, whereas competing models often over-smoothed these critical transitions.
  • Efficiency: Despite its mathematical complexity, EgPDE-Net maintains a small footprint (~50k parameters) and competitive inference times, making it practical for real-world deployment.

Conclusion

EgPDE-Net advances continuous-time forecasting by integrating exogenous variables through a PDE-based framework. By using external data to guide the derivative of the target state, the model achieves higher precision in arbitrary-step prediction. Future work will explore fully continuous encoding networks and methods to handle informative missingness in historical records.

Summary updated

Abstract

While exogenous variables have a major impact on performance improvement in time series analysis, inter-series correlation and time dependence among them are rarely considered in the present continuous methods. The dynamical systems of multivariate time series could be modelled with complex unknown partial differential equations (PDEs) which play a prominent role in many disciplines of science and engineering. In this paper, we propose a continuous-time model for arbitrary-step prediction to learn an unknown PDE system in multivariate time series whose governing equations are parameterised by self-attention and gated recurrent neural networks. The proposed model, \underline{E}xogenous-\underline{g}uided \underline{P}artial \underline{D}ifferential \underline{E}quation Network (EgPDE-Net), takes account of the relationships among the exogenous variables and their effects on the target series. Importantly, the model can be reduced into a regularised ordinary differential equation (ODE) problem with special designed regularisation guidance, which makes the PDE problem tractable to obtain numerical solutions and feasible to predict multiple future values of the target series at arbitrary time points. Extensive experiments demonstrate that our proposed model could achieve competitive accuracy over strong baselines: on average, it outperforms the best baseline by reducing $9.85\%$ on RMSE and $13.98\%$ on MAE for arbitrary-step prediction.
BibTeX
@article{gao2022egpde,
  author = {Penglei Gao and Xi Yang and Kaizhu Huang and Rui Zhang and Ping Guo and John Yannis Goulermas},
  title = {{EgPDE-Net: Building Continuous Neural Networks for Time Series Prediction with Exogenous Variables}},
  journal = {IEEE Trans. Cybernetics},
  year = {2022},
  doi = {10.1109/tcyb.2024.3364186},
}
Figure