Self-Recover: Forecasting Block Maxima in Time Series from Predictors with Disparate Temporal Coverage Using Self-Supervised Learning

Asadullah Hill Galib, Andrew McDonald, Pang-Ning Tan, Lifeng LuoIJCAI 2023 · DOI

Introduction

Predicting extreme events, such as the maximum intensity of a hurricane or record-breaking heatwaves, is a critical challenge for disaster management and climate resilience. In statistical terms, this involves forecasting the "block maxima"—the highest value a variable reaches within a defined time window. Traditional deep learning models often struggle with these "heavy-tail" events because they are typically optimized to predict the mean of a distribution rather than its extremes.

A significant hurdle in training effective models for extreme events is the "disparate temporal coverage" problem. While historical observations ($x^H$) may span many decades, high-fidelity physical or statistical model-based forecasts ($x^{M_o}$) are often only available for much shorter, more recent periods. This creates large gaps in the training data, as illustrated in Figure 1.

Disparate temporal coverage between historical observations and model-based forecasts.

The Self-Recover Framework

To address these challenges, the authors propose Self-Recover, a deep learning framework that fuses historical data with model-based simulations while handling missing values through self-supervised learning.

1. Self-Supervised Missing Value Imputation

The framework employs a self-supervised learning (SSL) strategy to "fill in" missing model-based forecasts. By using a Denoising Autoencoder (DAE) and Contrastive Learning, the model learns robust representations from the available historical data. It applies "jittering" (adding noise) to time-series data to create augmented views, forcing the encoder to learn features that are invariant to noise. These learned representations are then used to impute the missing values in the model-based outputs ($x^{M_f}$), ensuring a continuous data stream for downstream tasks.

2. Residual Learning for Data Fusion

Rather than simply concatenating historical and model-based features—which can lead to overfitting or slow convergence due to high dimensionality—Self-Recover utilizes a residual learning approach. The model-based data is treated as a refinement or "residual" to the historical observations: $$h_t^{(H+M)} = h_t^H + \alpha \times h_t^M$$ This architecture allows the network to prioritize the reliable historical signal while using the simulation data to correct and enhance the forecast.

3. GEV Distribution Integration

To provide statistically sound forecasts, Self-Recover integrates the Generalized Extreme Value (GEV) distribution. Instead of a point estimate, the network outputs the parameters of the GEV distribution: location ($\mu$), scale ($\sigma$), and shape ($\xi$). This enables the calculation of precise confidence intervals and quantiles, which are essential for assessing "worst-case" risks.

The Self-Recover architecture, featuring self-supervised imputation, GEV parameter estimation, and block maxima prediction.

Experimental Results

The framework was evaluated on two real-world datasets: hurricane intensity (HURDAT2) and regional climate temperatures (NARCCAP).

  • Hurricane Intensity: Self-Recover achieved a Root Mean Square Error (RMSE) of 14.88, outperforming standard Transformers (15.31) and LSTMs. Notably, for the most extreme Category 5 hurricanes, the model achieved an F1-score of 0.92, a significant improvement over raw model forecasts.
  • Climate Extremes: In temperature forecasting, the model reduced the RMSE to 2.79, compared to 3.49 for the previous state-of-the-art model, DeepExtrema.

Conclusion

Self-Recover demonstrates that self-supervised representation learning and residual data fusion can effectively bridge the gap between long-term historical records and short-term high-resolution simulations. By grounding deep learning in extreme value theory (GEV), the framework provides a robust tool for predicting the events that matter most for public safety and infrastructure planning. Future research will look into more advanced data augmentation and the incorporation of spatial-temporal data like satellite imagery.

Summary updated

BibTeX
@inproceedings{galib2023self,
  author = {Asadullah Hill Galib and Andrew McDonald and Pang-Ning Tan and Lifeng Luo},
  title = {{Self-Recover: Forecasting Block Maxima in Time Series from Predictors with Disparate Temporal Coverage Using Self-Supervised Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/414},
}
Figure