High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes

David Salinas, Michael Bohlke-Schneider, Laurent Callot, Roberto Medico, Jan Gasthaus

Abstract

Predicting the dependencies between observations from multiple time series is critical for applications such as anomaly detection, financial risk management, causal analysis, or demand forecasting. However, the computational and numerical difficulties of estimating time-varying and high-dimensional covariance matrices often limits existing methods to handling at most a few hundred dimensions or requires making strong assumptions on the dependence between series. We propose to combine an RNN-based time series model with a Gaussian copula process output model with a low-rank covariance structure to reduce the computational complexity and handle non-Gaussian marginal distributions. This permits to drastically reduce the number of parameters and consequently allows the modeling of time-varying correlations of thousands of time series. We show on several real-world datasets that our method provides significant accuracy improvements over state-of-the-art baselines and perform an ablation study analyzing the contributions of the different components of our model.

Introduction

In many real-world applications—ranging from financial risk management to retail demand forecasting and compute cluster monitoring—time series are rarely independent. They exhibit complex, time-varying correlations that are crucial for accurate probabilistic forecasting. However, modeling these dependencies in high-dimensional settings, where the number of series $N$ can reach thousands, presents significant challenges.

Traditional multivariate methods, such as Vector Autoregression (VAR) or Multivariate GARCH, suffer from the "curse of dimensionality." The number of parameters in a full covariance matrix scales as $O(N^2)$, making them computationally prohibitive beyond a few hundred dimensions. Furthermore, real-world data often features non-Gaussian distributions and disparate scales, which standard models fail to normalize effectively.

Methodology

The proposed architecture addresses these issues by merging deep learning with classical copula theory through four key components:

  1. RNN Backbone: An LSTM-RNN processes each time series to capture temporal dynamics. Parameters are shared across all series to learn global patterns while maintaining scalability.
  2. Gaussian Copula: To handle non-Gaussian data and varying scales, the model estimates the marginal distribution of each series using a non-parametric empirical Cumulative Distribution Function (CDF). This maps arbitrary distributions into a standard Normal space, effectively "equalizing" the series regardless of their original magnitude.
  3. Low-Rank Covariance Structure: Instead of a full $N \times N$ matrix, the covariance is parametrized as a "low-rank plus diagonal" structure ($\Sigma = D + VV^T$). This reduces the parameter count from $O(N^2)$ to $O(N \times r)$, where $r$ is a small rank (e.g., 10).
  4. Joint Learning: The RNN's hidden state at each time step predicts the parameters of this low-rank distribution ($\mu, D, V$), allowing the model to learn how correlations evolve over time.

Experimental Setup

The model was evaluated on six diverse real-world datasets with dimensions ranging from 8 to 2,000:

  • Exchange rate: Daily rates of 8 currencies.
  • Solar: Hourly production from 137 stations.
  • Electricity: Hourly consumption of 370 customers.
  • Traffic: Hourly occupancy rates of 963 car lanes.
  • Taxi: 30-minute traffic intervals for 1,214 NYC locations.
  • Wikipedia: Daily page views for 2,000 pages.

Results and Discussion

The proposed GP-Copula model significantly outperformed baselines such as VAR, GARCH, and independent RNNs. It improved the Continuous Ranked Probability Score (CRPS) by an average of 10% over the next best model, with a 40% improvement in the aggregated CRPS-Sum metric.

Learned correlation matrix for the NYC Taxi dataset at 14:30.

A key advantage of the model is its ability to recover meaningful dependencies. For the NYC Taxi dataset, the predicted correlation matrices clearly showed locations that were geographically close or linked by traffic flow. These relationships evolved logically over the course of a day, reflecting changing urban dynamics.

Spatial visualization of high correlations between NYC taxi locations.

Efficiency gains were also substantial. On the Traffic dataset, the GP-Copula required only 44K parameters, compared to 1.1M for a low-rank Vec-LSTM and 38M for a full-rank version. This efficiency stems from the low-rank assumption and the ability to train on random subsets of dimensions.

Conclusion

By decoupling marginal distributions from dependencies using a low-rank Gaussian copula, this framework enables scalable and accurate multivariate forecasting in high-dimensional spaces. The model successfully captures time-varying correlations with a fraction of the parameters required by traditional methods. Future work may explore handling non-stationary marginals and automated selection of the rank hyperparameter.

Summary updated

Abstract

Predicting the dependencies between observations from multiple time series is critical for applications such as anomaly detection, financial risk management, causal analysis, or demand forecasting. However, the computational and numerical difficulties of estimating time-varying and high-dimensional covariance matrices often limits existing methods to handling at most a few hundred dimensions or requires making strong assumptions on the dependence between series. We propose to combine an RNN-based time series model with a Gaussian copula process output model with a low-rank covariance structure to reduce the computational complexity and handle non-Gaussian marginal distributions. This permits to drastically reduce the number of parameters and consequently allows the modeling of time-varying correlations of thousands of time series. We show on several real-world datasets that our method provides significant accuracy improvements over state-of-the-art baselines and perform an ablation study analyzing the contributions of the different components of our model.
BibTeX
@inproceedings{salinas2019high,
  author = {David Salinas and Michael Bohlke-Schneider and Laurent Callot and Roberto Medico and Jan Gasthaus},
  title = {{High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=F9juRXbm-6},
}
Figure