Synergetic Learning of Heterogeneous Temporal Sequences for Multi-Horizon Probabilistic Forecasting

Longyuan Li, Jihai Zhang, Junchi Yan, Yaohui Jin, Yunhao Zhang, Yanjie Duan, Guangjian Tian

Introduction

In complex real-world systems, temporal data typically manifests in two distinct, interrelated formats: synchronous time-series (regularly sampled data like hourly temperature) and asynchronous event sequences (irregular occurrences like storms or traffic accidents). While these data types are deeply coupled—for instance, extreme weather events directly impact energy consumption—most existing forecasting models treat them in isolation or through coarse approximations that discard fine-grained timing information.

The Variational Synergetic Multi-Horizon Network (VSMHN) addresses this gap. It is a novel deep generative model designed to "synergize" these heterogeneous data types within a unified framework, achieving state-of-the-art multi-horizon probabilistic forecasting by explicitly modeling the interactions between continuous trends and discrete shocks.

Methodology: The VSMHN Architecture

Built upon the Conditional Variational Autoencoder (CVAE) framework, the VSMHN architecture is designed to handle the unaligned nature of heterogeneous sequences.

The VSMHN architecture featuring dual RNN encoders, an Auxiliary Transition Unit for synchronization, and a generative decoder for probabilistic forecasting.

Time-Aware Hybrid Encoding

The model utilizes two specialized Recurrent Neural Networks (RNNs) to process historical data:

  • Time-series RNN: Operates at fixed intervals to capture regular seasonal and trend components.
  • Event RNN: Transits only upon event occurrences, encoding the specific timing ((\Delta t)) and type of asynchronous shocks.

Synchronization and Latent Synergy

A critical innovation is the Auxiliary Transition Unit, which forces the Event RNN to perform a final synchronization step at the end of the observation window. This allows for efficient batch training despite irregular event timings. The resulting hidden states are processed by a Synergetic Layer, which maps the combined features into a latent Gaussian distribution representing the "hidden state of the world."

Stochastic Generation

To provide probabilistic forecasts, the decoder employs Monte-Carlo sampling. By drawing samples from the latent space and passing them through a stochastic RNN, the model generates a range of potential future trajectories, allowing for the estimation of confidence intervals and uncertainty.

Experimental Results and Evaluation

The VSMHN was evaluated across three diverse real-world domains: Electricity usage, Traffic occupancy, and Environmental air quality (PM2.5, temperature, etc.). In the Environmental dataset, the model utilized true exogenous events such as the start and stop of rain and wind.

Performance comparison of VSMHN against baseline models across six variables, showing RMSE and CRPS metrics.

Key Findings

  • Superior Accuracy: VSMHN consistently outperformed established baselines like DeepAR, MQRNN, and traditional statistical models (VAR, SARIMA) in terms of Root Mean Square Error (RMSE) and Continuous Ranked Probability Score (CRPS).
  • The Value of Synergy: Ablation studies showed that a version of the model using only time-series data (VSMHN-TS) performed significantly worse, confirming that explicit event modeling is the primary driver of performance gains.
  • Uncertainty Calibration: The model demonstrated superior calibration, providing more realistic confidence intervals compared to deterministic or poorly calibrated probabilistic models.

Discussion and Future Work

The VSMHN proves that events are not merely external noise but are integral to understanding system dynamics. By preserving high-frequency timing information, the model effectively handles "one-to-many" mappings where identical past trends might lead to different futures depending on intervening events.

Future research directions include refining event definitions for datasets where events are currently derived from the time-series itself and exploring non-Gaussian observation models to better handle skewed or multi-modal future distributions. Additionally, optimizing the computational overhead of dual-RNN encoding and intensive Monte-Carlo sampling remains a priority for real-time applications.

Summary updated

Abstract

Time-series is ubiquitous across applications, such as transportation, finance and healthcare. Time-series is often influenced by external factors, especially in the form of asynchronous events, making forecasting difficult. However, existing models are mainly designated for either synchronous time-series or asynchronous event sequence, and can hardly provide a synthetic way to capture the relation between them. We propose Variational Synergetic Multi-Horizon Network (VSMHN), a novel deep conditional generative model. To learn complex correlations across heterogeneous sequences, a tailored encoder is devised to combine the advances in deep point processes models and variational recurrent neural networks. In addition, an aligned time coding and an auxiliary transition scheme are carefully devised for batched training on unaligned sequences. Our model can be trained effectively using stochastic variational inference and generates probabilistic predictions with Monte-Carlo simulation. Furthermore, our model produces accurate, sharp and more realistic probabilistic forecasts. We also show that modeling asynchronous event sequences is crucial for multi-horizon time-series forecasting.
BibTeX
@inproceedings{li2021synergetic,
  author = {Longyuan Li and Jihai Zhang and Junchi Yan and Yaohui Jin and Yunhao Zhang and Yanjie Duan and Guangjian Tian},
  title = {{Synergetic Learning of Heterogeneous Temporal Sequences for Multi-Horizon Probabilistic Forecasting}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i10.17023},
}
Figure