TL;DR
This paper introduces NEC+, a novel framework for time series prediction that specifically addresses the challenge of accurately forecasting extreme events, which are often underrepresented in training data. By employing separate models for normal and extreme conditions and a sophisticated classification mechanism, NEC+ significantly outperforms existing methods in hydrologic prediction tasks.
What Problem Does This Paper Address?
Forecasting time series data is crucial across many domains, but it becomes particularly challenging when the data exhibits extreme events. Hydrologic prediction, for instance, often involves water levels that can fluctuate dramatically due to factors like weather and human activity. Traditional and even many deep learning models struggle with these imbalanced datasets, where rare extreme events can be overlooked during training, leading to poor predictions when they matter most. Existing methods often evaluate performance on well-behaved data, failing to demonstrate their ability to generalize to scenarios with significant outliers. This paper aims to bridge this gap by developing a model that can adapt to and accurately predict both normal and extreme values in time series data.
Methods
The core of the proposed solution is the NEC+ framework, which comprises three parallel models:
- Normal (N) Model: Trained to predict typical, non-extreme values.
- Extreme (E) Model: Specifically trained to predict rare, extreme values.
- Classifier (C) Model: Determines whether a future prediction falls into the "normal" or "extreme" category.
Key innovations within NEC+ include:
- Probability-Enhanced LSTM: The framework leverages LSTM networks, known for their strength in sequence modeling, but enhances them with probability-based features.
- Gaussian Mixture Model (GMM) Indicator: A GMM is used to generate an indicator feature that captures the distribution of the time series. This indicator is fed as an additional input to the N and E models, helping them differentiate between normal and extreme conditions.
- Selective Backpropagation: To prevent models from being unduly influenced by the majority class (normal values for the E model, and vice-versa), backpropagation is selectively applied. The N model's loss is only calculated on normal values, and the E model's loss on extreme values.
- Parameterized Loss Function: A custom loss function for the C model is introduced, which uses a tunable parameter to control the emphasis on predicting extreme events, addressing the class imbalance problem.
- Exogenous Variables: The framework can incorporate external data (like rainfall) as exogenous variables to improve predictions, particularly for reservoirs influenced by upstream conditions.
- Two-Level Sampling Policy: To handle the sparsity of extreme events, a stratified sampling approach is used, allowing the E and C models to oversample regions with extreme values.
Datasets
The study utilizes a dataset comprising approximately 31 years of hourly reservoir water level sensor data from 9 reservoirs in Santa Clara County, California. This data is supplemented with hourly rainfall data from 5 rain gauge sensors in the same region. The dataset is described as having a significant imbalance between normal and extreme values, with extreme events ranging from 0.08% to 4.08% of the time series.
Key Insights and Ideas
- Extreme events require specialized treatment: The paper strongly suggests that simply applying standard time series models, even advanced deep learning ones, to imbalanced data will lead to suboptimal performance for extreme events. The NEC+ framework's success hinges on its ability to treat normal and extreme conditions separately.
- GMM indicator is crucial for NEC+: While adding a GMM indicator to a baseline LSTM model provided only marginal improvements, it significantly boosted the performance of the NEC framework. This highlights the importance of explicitly providing distributional information to the adaptive components of NEC+.
- Selective backpropagation is key to model focus: By forcing the N model to learn from normal data and the E model from extreme data, the framework ensures each component focuses on its intended task without being diluted by the majority class.
- Oversampling alone is insufficient: The experiments showed that while oversampling can help reduce the RMSE for extreme values, it can lead to worse overall performance. NEC+'s approach of separate models and targeted training is more effective than simple oversampling.
- Exogenous variables benefit NEC+ more than baseline LSTM: While exogenous variables (like rainfall) did not consistently improve the baseline LSTM, they provided significant benefits to the NEC+ model, suggesting that the adaptive nature of NEC+ can better leverage external signals.
Key Results
- Superior performance: NEC+ significantly outperforms state-of-the-art baselines, including ARIMA, Prophet, LSTM, DNN-U, A-LSTM, and N-BEATS, across all 9 reservoirs.
- RMSE improvements: On average, NEC+ achieved 37% and 33% better RMSE than DNN-U and A-LSTM, respectively. It also outperformed N-BEATS on 8 out of 9 sensors.
- Adaptability to diverse distributions: The proposed architecture and hyperparameters generalized well across different reservoir datasets, indicating robustness.
- Effective prediction of extreme events: Visualizations and quantitative results demonstrate NEC+'s ability to more closely follow ground truth, especially during extreme fluctuations.
Limitations and Future Work
- Hyperparameter tuning: While the paper notes that hyperparameters were similar across sensors, some tuning was still required, and the optimal parameters can vary.
- Generalizability to other domains: The study focuses on hydrologic prediction. Further research could explore the applicability of NEC+ to other domains with imbalanced time series data.
- Alternative indicator methods: The paper suggests that other statistical methods like quantile or expectile information, or different transformation skills, could be used in place of the GMM indicator.
- Framework flexibility: The N, E, and C models can be implemented using different neural network architectures, offering avenues for future exploration and improvement.
Summary updated