Abstract
Time series models often deal with extreme events and anomalies, both prevalent in real-world datasets. Such models often need to provide careful probabilistic forecasting, which is vital in risk management for extreme events such as hurricanes and pandemics. However, it is challenging to automatically detect and learn to use extreme events and anomalies for large-scale datasets, which often require manual effort. Hence, we propose an anomaly-aware forecast framework that leverages the previously seen effects of anomalies to improve its prediction accuracy during and after the presence of extreme events. Specifically, the framework automatically extracts anomalies and incorporates them through an attention mechanism to increase its accuracy for future extreme events. Moreover, the framework employs a dynamic uncertainty optimization algorithm that reduces the uncertainty of forecasts in an online manner. The proposed framework demonstrated consistent superior accuracy with less uncertainty on three datasets with different varieties of anomalies over the current prediction models.
Introduction
Standard time series forecasting models, such as LSTMs or Transformers, frequently struggle when encountering extreme events. These "black swan" occurrences—like hurricanes or pandemics—are rare and can cause massive structural shifts in data. Conventional models often treat these anomalies as noise to be smoothed over, yet in risk management and disaster response, these events are often the most critical data points. AA-Forecast addresses this gap by creating a model that automatically detects these events, learns from their historical impacts, and provides high-confidence forecasts during periods of volatility.
The AA-Forecast Framework
The AA-Forecast framework is built upon three core innovations designed to handle the complexity of extreme events.
1. STAR Decomposition
Unlike standard additive decomposition methods (Trend + Seasonal + Residual), AA-Forecast introduces STAR Decomposition, a multiplicative approach: $x = s \times t \times a \times r$. This method explicitly extracts an Anomaly ($a$) component using a robustness score based on the median of residuals. By separating true anomalies from random noise, the model can treat past disasters as informative features rather than outliers.
2. Anomaly-Aware Attention
To manage long-term dependencies without the quadratic computational costs of full Transformers, the model utilizes an RNN backbone (LSTM or GRU) supplemented by a "spotlight" attention mechanism. This mechanism selectively activates only for time steps where an extreme event or anomaly was previously detected. This allows the model to "remember" the system's reaction to a specific historical event—such as a major hurricane years prior—without being overwhelmed by intervening normal data.
3. Dynamic Uncertainty Optimization
While most models use a static dropout rate, AA-Forecast treats dropout as a tunable parameter during the inference phase. It employs an online optimization algorithm to find the specific dropout probability that minimizes forecast variance at each time step, leading to more "confident" predictions during high-volatility periods.
Experimental Evaluation
The framework was evaluated on three diverse datasets:
- Hurricane: A new spatio-temporal benchmark tracking monthly sales revenue across 67 Florida counties (2003–2020), aligned with historical hurricane data.
- COVID-19: US employee counts during the pandemic, including state-level business closure policies.
- Electricity: A public benchmark of hourly electricity consumption for 370 consumers.
Key Results
AA-Forecast was compared against several baselines, including ARIMA, UberNN, and the Transformer-based TSE-SC.
- Superior Accuracy: The framework consistently achieved the lowest Continuous Ranked Probability Score (CRPS) and Root Mean Square Error (RMSE) across all datasets.
- Reduced Uncertainty: The dynamic dropout optimization significantly lowered the standard deviation in predictions, particularly during extreme events.
- Zero-Shot Performance: The model demonstrated an ability to transfer knowledge about hurricane impacts to counties it had never seen during training, showcasing strong generalization.
Conclusion
AA-Forecast shifts the paradigm of time series forecasting by treating anomalies as structural lessons rather than noise. By combining multiplicative decomposition with selective attention and online uncertainty tuning, the framework provides a robust tool for risk management in an increasingly volatile world. Future work will explore targeting specific groups of neurons during dropout optimization to further refine the model's handling of complex dynamics.
Summary updated