TL;DR
EgPDE-Net is a novel continuous-time neural network that models multivariate time series as a system of Partial Differential Equations (PDEs). By mathematically reducing a complex PDE problem into a tractable, regularized Ordinary Differential Equation (ODE) framework, it enables high-accuracy forecasting at any arbitrary time point while effectively capturing how exogenous variables drive the target series.
What Problem Does This Paper Address?
Traditional deep learning models for time series, such as Recurrent Neural Networks (RNNs), are fundamentally discrete—they make predictions at fixed intervals (e.g., $T+1, T+2$). This breaks down when we need arbitrary-step prediction, such as forecasting a value at $T+1.5$ when data is only sampled at integer steps.
While "Neural ODEs" were introduced to create continuous-time models, they often struggle with multivariate data. Specifically, they rarely account for the complex inter-series correlations and time-dependencies between exogenous variables (external factors like weather) and the target series (the variable we want to predict, like electricity demand). The authors argue that these systems are better described by PDEs, which can model changes across multiple independent variables simultaneously, but PDEs are notoriously difficult to solve numerically within a neural network.
Methods
The core contribution is the Exogenous-guided Partial Differential Equation Network (EgPDE-Net). The authors propose a framework that treats the multivariate system as a reaction-diffusion PDE. To make this solvable, they introduce a mathematical "bridge" (Lemma III.1) that transforms the PDE into two coupled ODE pipelines:
- The Exogenous Pipeline: Exogenous variables are processed through a self-attention block to capture global and local correlations. A first ODE solver then generates "partial differential weights"—essentially a dynamic map of how these external factors should guide the target series.
- The Target Pipeline: The target series is processed via a GRU to capture its internal temporal patterns. A second ODE solver then generates the final latent trajectory, using the weights from the first pipeline as a regularization term to "steer" the prediction.
By integrating these ODEs, the model can produce a prediction for any continuous time $t$, not just fixed intervals.
Datasets
The model was evaluated on four real-world multivariate datasets:
- SML2010: Indoor temperature forecasting (13 exogenous variables).
- Electricity: Hourly electricity consumption in the US (15 weather-related exogenous variables).
- ETTh1 & ETTh2: Electricity Transformer Temperature (6 power load features).
To test arbitrary-step prediction, the authors re-sampled these datasets to half their original frequency during training and tasked the models with predicting the "missing" intermediate points during testing.
Key Insights and Ideas
- PDEs as a Natural Fit: While ODEs only describe evolution over time, PDEs allow the model to learn derivatives with respect to both time and the exogenous variable space. This provides a more rigorous mathematical foundation for multivariate forecasting.
- Guidance as Regularization: The authors treat the influence of exogenous variables as a "regularized guidance" term. This prevents the model from drifting and ensures the predicted trajectory is physically and logically consistent with external drivers.
- Tractability via Reduction: The "aha moment" is the realization that a complex PDE can be approximated by two neural networks acting as "weak solutions." This avoids the computational nightmare of traditional numerical PDE solvers while retaining the model's continuous nature.
- Interpretability: The use of self-attention allows researchers to see which exogenous variables (e.g., "FeelsLike" temperature vs. "Humidity") are most influential at specific times.
Key Results
EgPDE-Net consistently outperformed both discrete (LSTNet, IMV-tensor) and continuous (Latent ODE, ETN-ODE) baselines.
- Arbitrary-Step Accuracy: On average, it reduced RMSE by 9.85% and MAE by 13.98% compared to the best-performing baseline.
- Standard Multi-Step Prediction: Even on fixed-interval tasks ($M=1, 5, 10$), EgPDE-Net achieved the lowest errors in nearly all categories across all four datasets.
- Efficiency: Despite the complexity of the PDE formulation, the model maintains a competitive inference time and a relatively small parameter footprint (e.g., ~50k parameters for the Electricity dataset).
Limitations and Future Work
The authors acknowledge a few areas for improvement:
- Computational Overhead: Continuous models, by nature, require more training and inference time than simple discrete networks due to the ODE integration steps.
- Discrete Encoding: The current encoding layers (GRU and Self-Attention) are still discrete. The authors suggest exploring fully continuous encoding networks in the future.
- Missing Data: While the model handles arbitrary time steps, it does not yet explicitly model "informative missingness" (where the fact that data is missing carries its own meaning), which is a common challenge in clinical and sensor data.
Summary updated