POLA: Online Time Series Prediction by Adaptive Learning Rates

Wenyu Zhang

This paper presents POLA (Predicting Online by Learning rate Adaptation), a framework designed to make deep learning models more resilient and adaptive when forecasting streaming time series data.

TL;DR

POLA is a meta-learning framework that automatically tunes the learning rate of recurrent neural networks (RNNs) in real-time. By treating the most recent data batch as a mini "test-then-train" scenario, it allows models to adapt rapidly to changing data distributions (concept drift) without the need for manual scheduling or computationally expensive second-order derivatives.

What Problem Does This Paper Address?

In many real-world scenarios—like financial forecasting, traffic monitoring, or energy grid management—data arrives in a continuous stream. A major challenge in these environments is concept drift, where the underlying patterns in the data change over time. A model trained on last month's data might be completely obsolete today.

While online learning techniques exist to update models on-the-fly, they often struggle with a "Goldilocks" problem:

  1. Too slow: A low learning rate prevents the model from adapting to sudden shifts.
  2. Too fast: A high learning rate causes the model to overfit to recent noise or outliers.
  3. Too heavy: Advanced methods that calculate the "optimal" step size often require Hessian matrices (second-order derivatives), which are computationally prohibitive for deep neural networks with many parameters.

Methods

The authors propose a bilevel optimization approach called POLA. Instead of using a fixed or heuristically scheduled learning rate, POLA "learns how to learn" from the data stream itself.

The Core Mechanism: When a new batch of data arrives, POLA doesn't just perform a standard gradient update. Instead, it performs a meta-learning step:

  1. The Split: It divides the current data batch into two parts: a meta-training set (older samples in the batch) and a meta-validation set (the most recent samples).
  2. The Simulation: It simulates a model update on the meta-training set using different potential learning rates.
  3. The Optimization: It selects the learning rate that results in the lowest error on the meta-validation set. This "assimilates" the evaluation process into the training process.

The authors provide two implementations:

  • POLA-FS (Finite Search): Picks the best learning rate from a pre-defined set of candidates.
  • POLA-GD (Gradient Descent): Uses gradient descent to optimize the learning rate itself, allowing for a more continuous and fine-grained adaptation.

To ensure stability, the final learning rate used for the actual model update is a moving average of these meta-learned values, preventing the model from overreacting to a single noisy batch.

Datasets

The researchers evaluated POLA on two real-world datasets known for exhibiting concept drift:

  • Sunspot Dataset: Monthly sunspot numbers from 1749 to 2020. This dataset features long-term cyclical patterns (roughly 11-year cycles) that are notoriously difficult to predict perfectly.
  • Household Power Consumption: A multivariate dataset tracking electricity usage (active power, intensity, voltage). This represents a high-frequency, noisy real-world environment.

Key Insights and Ideas

  • Data-Driven Scheduling: Unlike RMSprop or Adam, which scale learning rates based on historical gradient magnitudes, POLA scales them based on actual predictive performance on the most recent data.
  • Efficiency through Approximation: By using a first-order Taylor expansion in the bilevel optimization, the authors avoid the "Hessian bottleneck," making meta-learning feasible for deep RNNs in streaming settings.
  • The "Useful Update" Intuition: The meta-learning factor ($\beta$) acts as a gatekeeper. If the current batch of data is noisy or inconsistent with the immediate future, POLA automatically shrinks the learning rate, effectively "waiting" for more reliable data before making significant model changes.

Key Results

POLA was tested across three architectures: standard RNNs, LSTMs, and GRUs.

  • Superior Accuracy: In almost all tests, POLA (especially the GD version) achieved the lowest Normalized Root-Mean-Square Error (RMSE) compared to baselines like MAML, RMSprop, and "Follow The Leader" (FTL) strategies.
  • Robustness to Drift: On the Sunspot dataset, POLA-GD achieved an RMSE of 0.500, significantly outperforming the pre-trained static model (0.572) and standard Online-SGD (0.552).
  • Architecture Agnostic: The method consistently improved performance regardless of whether an LSTM or GRU was used, suggesting the learning rate adaptation is a fundamental improvement for online time series tasks.

Limitations and Future Work

  • Computational Cost: While POLA avoids Hessians, the meta-learning step still adds overhead. POLA-GD can be significantly slower than standard SGD (e.g., 40s vs 3.5s for the Sunspot dataset), which might be a factor for ultra-low-latency applications.
  • Batch Size Sensitivity: The method relies on splitting a batch for meta-validation. If the available batch size is very small (e.g., $b < 10$), the meta-learning estimate becomes less accurate.
  • Future Directions: The authors suggest exploring more sophisticated ways to split the meta-data or applying the framework to even larger, more complex multivariate forecasting problems.

Summary updated

Abstract

Online prediction for streaming time series data has practical use for many real-world applications where downstream decisions depend on accurate forecasts for the future. Deployment in dynamic environments requires models to adapt quickly to changing data distributions without overfitting. We propose POLA (Predicting Online by Learning rate Adaptation) to automatically regulate the learning rate of recurrent neural network models to adapt to changing time series patterns across time. POLA meta-learns the learning rate of the stochastic gradient descent (SGD) algorithm by assimilating the prequential or interleaved-test-then-train evaluation scheme for online prediction. We evaluate POLA on two real-world datasets across three commonly-used recurrent neural network models. POLA demonstrates overall comparable or better predictive performance over other online prediction methods.
BibTeX
@inproceedings{zhang2021pola,
  author = {Wenyu Zhang},
  title = {{POLA: Online Time Series Prediction by Adaptive Learning Rates}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=31q2QJo-Nsp},
}

Loading related content...

Figure