Self-adaptive Extreme Penalized Loss for Imbalanced Time Series Prediction

Yiyang Wang, Yuchen Han, Yuhan GuoIJCAI 2024

Introduction

Accurate forecasting of extreme events—such as storm surges, wind gusts, or air pollution spikes—is critical for disaster mitigation and resource management. However, standard deep learning models often struggle with these "tail" events due to data imbalance. Conventional loss functions like Mean Squared Error (MSE) prioritize average performance, leading to "conservative" models that systematically underestimate high-impact extremes. This paper introduces the Self-adaptive Extreme Penalized Loss (EPL) framework, a novel approach designed to capture extreme fluctuations without compromising baseline accuracy.

Methodology: The EPL Framework

The proposed framework addresses the limitations of MSE through two primary innovations: a tripartite loss function and a decomposition-based architecture.

1. Extreme Penalized Loss (EPL)

The core of the methodology is the EPL function, which moves beyond binary (normal vs. extreme) classifications. It employs a tripartite division of the error space:

  • Normal Range: Errors are penalized using standard MSE to maintain baseline stability.
  • Extreme Underestimation: A heavy exponential penalty is applied to "missed" extremes, forcing the model to take these high-impact points seriously.
  • Extreme Overestimation: Penalties are relaxed for over-predictions in the extreme range. This asymmetry reflects real-world utility: in scenarios like flood warnings, a slight over-prediction is significantly less costly than a failure to predict a catastrophic event.
The tripartite EPL loss function and adaptive thresholding mechanism.

The model further utilizes Self-Adaptive Hyperparameters. It automatically calculates the extreme threshold ($\gamma$) and penalty strength ($\lambda$) by analyzing the statistical distribution of the training data, ensuring the model remains robust across different datasets.

2. Decomposition Architecture

To prevent the extreme-focused loss from distorting normal predictions, the system employs a "divide and conquer" strategy using Empirical Wavelet Transform (EWT). The time series is decomposed into two distinct components:

  • Long-term Trend ($u_t$): A smooth component representing the baseline, predicted via an Attention-LSTM optimized with MSE.
  • Residual Variation ($v_t$): The volatile component containing spikes and fluctuations, predicted via an LSTM optimized with the new EPL loss.
The complete EPL framework architecture, featuring EWT decomposition and dual-path LSTM prediction.

Experimental Evaluation

The framework was rigorously tested against eight real-world datasets spanning wave height (BSG, DCN, NJI, XMD, Kaggle-WH), wind speed (Kaggle-WS), and air quality (Beijing, Italy). It was compared against state-of-the-art baselines including ARIMA, N-Beats, Transformers, and specialized extreme-value models like SADI and NEC+.

Key Results

  • Superior Accuracy: The EPL framework consistently achieved the lowest Root Mean Square Error (RMSE) and highest $R^2$ values across nearly all test cases.
  • Extreme Event Precision: The model demonstrated significantly higher $F_1$ scores for extreme event detection compared to baselines, indicating a superior ability to correctly identify and quantify peaks.
  • Predictive Power for Maxima: When evaluating "Block Maxima" (the peak value within a specific time window), the framework showed a 20% improvement in $R^2$ over specialized models like DeepExtrema.
  • Ablation Success: Tests confirmed that both the Attention module and the EWT decomposition are essential for maintaining the balance between trend accuracy and extreme sensitivity.

Conclusion

The Self-adaptive EPL framework provides a robust solution to the "conservative prediction" problem in time series forecasting. By decoupling trends from variations and applying an asymmetric, tripartite loss function, the model successfully captures high-impact extremes while preserving high accuracy for normal data. While currently optimized for one-sided positive extremes, future iterations may extend this logic to two-sided extreme scenarios, such as temperature fluctuations.

Summary updated

BibTeX
@inproceedings{wang2024self,
  author = {Yiyang Wang and Yuchen Han and Yuhan Guo},
  title = {{Self-adaptive Extreme Penalized Loss for Imbalanced Time Series Prediction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure