DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks

Valentin Flunkert, David Salinas, Jan GasthausarXiv 2017 · OpenReview

TL;DR

This paper introduces DeepAR, a novel deep learning approach for probabilistic time series forecasting that leverages information from thousands of related time series to improve accuracy. By training a single autoregressive recurrent network on a large dataset, DeepAR can generate calibrated probabilistic forecasts, outperforming traditional methods by around 15% on real-world datasets.

What Problem Does This Paper Address?

Forecasting is crucial for optimizing business operations, from inventory management to resource allocation. Traditionally, forecasting methods focused on individual time series, requiring manual model selection and parameter tuning for each. However, modern applications often involve forecasting millions of related time series (e.g., product demand for a large retailer, energy consumption of households). In such scenarios, leveraging the collective information from similar time series can lead to more accurate and robust forecasts. The challenge lies in developing models that can effectively learn from this vast amount of related data, handle diverse scales and patterns, and produce probabilistic forecasts that capture uncertainty.

Methods

DeepAR employs an autoregressive recurrent neural network (RNN) architecture, specifically using Long Short-Term Memory (LSTM) cells. The core idea is to train a global model that learns from all time series simultaneously, rather than fitting individual models.

Here's a breakdown of the key methodological aspects:

  • Autoregressive Recurrent Network: The model predicts future values based on past observations and its own previous hidden states. This allows it to capture temporal dependencies.
  • Probabilistic Forecasting: Instead of just predicting a single point value, DeepAR estimates the parameters of a probability distribution for future values. This is crucial for decision-making under uncertainty.
  • Negative Binomial Likelihood: For count data (like demand), a negative binomial distribution is used, which is more appropriate than a Gaussian distribution for modeling count data with varying variances.
  • Handling Scale Differences: A significant challenge is the wide variation in magnitudes across time series. DeepAR addresses this by:
  • Item-dependent scaling: Dividing autoregressive inputs by an item-specific scale factor and scaling likelihood parameters accordingly.
  • Weighted sampling: Sampling training instances non-uniformly, giving higher probability to time series with larger scales to counteract underfitting.
  • Feature Engineering: The model can incorporate item-specific and time-dependent covariates (e.g., day of the week, month of the year, item age) which are learned via embeddings.

Datasets

The paper evaluates DeepAR on five datasets:

  • parts: 1046 aligned time series of monthly sales for automobile company items.
  • electricity: Hourly time series of electricity consumption for 370 customers.
  • traffic: Hourly occupancy rates for 963 car lanes in San Francisco.
  • ec and ec-sub: Weekly item sales from Amazon, characterized by diverse and erratic time series with a power-law distribution of item velocities.

The "ec" and "ec-sub" datasets are particularly noteworthy as they exhibit the power-law distribution of item velocities, which poses a significant challenge for traditional forecasting methods.

Key Insights and Ideas

  • Leveraging Collective Intelligence: The most impactful idea is that a single model trained on many related time series can learn more generalizable patterns and achieve higher accuracy than individual models, especially for time series with limited historical data.
  • Probabilistic Forecasts are Key: Moving beyond point forecasts to probabilistic ones is essential for robust decision-making, allowing businesses to quantify risk and optimize inventory levels more effectively.
  • Scale Matters (and Needs Handling): The paper highlights the critical issue of scale differences in large datasets and proposes practical solutions (scaling and weighted sampling) that significantly improve performance.
  • Deep Learning for Forecasting is Viable: Contrary to some prior mixed results, this work demonstrates that deep learning, specifically RNNs with appropriate likelihoods and scaling mechanisms, can effectively address complex forecasting problems.

Key Results

  • Accuracy Improvements: DeepAR achieved accuracy improvements of around 15% compared to state-of-the-art methods on several real-world datasets.
  • Handling Diverse Data: The model demonstrated strong performance across datasets with varying characteristics, including intermittent demand and power-law distributed item velocities.
  • Calibrated Uncertainty: Figures and metrics (like coverage) show that DeepAR produces well-calibrated probabilistic forecasts, meaning the predicted quantiles accurately reflect the true distribution of future values.
  • Efficient Training and Prediction: The model can be trained and used for prediction relatively quickly, even on large datasets, with a full run on a large dataset completed in under 10 hours on a single AWS instance.

Limitations and Future Work

  • Missing Observations: While the paper notes that DeepAR can handle missing observations in a principled way, experimental results for this setting were omitted due to the need for non-standard adjusted metrics. This is a clear area for future work.
  • Hyperparameter Tuning: While the paper claims "little or no hyperparameter tuning," some grid-search was performed for item output embedding dimension and LSTM nodes. Further exploration into automated hyperparameter optimization could be beneficial.
  • Generalizability: While the paper shows good performance across various datasets, further testing on even more diverse and challenging forecasting scenarios would be valuable.
  • Interpretability: As with many deep learning models, understanding why DeepAR makes certain predictions can be challenging, and further work on interpretability could be beneficial.

Summary updated

Abstract

Probabilistic forecasting, i.e. estimating the probability distribution of a time series' future given its past, is a key enabler for optimizing business processes. In retail businesses, for example, forecasting demand is crucial for having the right inventory available at the right time at the right place. In this paper we propose DeepAR, a methodology for producing accurate probabilistic forecasts, based on training an auto regressive recurrent network model on a large number of related time series. We demonstrate how by applying deep learning techniques to forecasting, one can overcome many of the challenges faced by widely-used classical approaches to the problem. We show through extensive empirical evaluation on several real-world forecasting data sets accuracy improvements of around 15% compared to state-of-the-art methods.
BibTeX
@misc{flunkert2017deepar,
  author = {Valentin Flunkert and David Salinas and Jan Gasthaus},
  title = {{DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks}},
  year = {2017},
  eprint = {1704.04110},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1704.04110},
}
Figure