MVG-CRPS: A Robust Loss Function for Multivariate Probabilistic Forecasting

Vincent Zhihao Zheng, Lijun SunSubmission to ICLR 2026 · Rejected

TL;DR

MVG-CRPS is a novel, strictly proper loss function for multivariate Gaussian probabilistic forecasting that combines the outlier robustness of the Continuous Ranked Probability Score (CRPS) with the analytical speed of closed-form expressions. By decorrelating multivariate predictions via PCA whitening, it reduces complex multivariate scoring to simple univariate CRPS calculations—eliminating the slow, noisy Monte Carlo sampling required by the Energy Score while remaining far less vulnerable to extreme values than negative log-likelihood.

What Problem Does This Paper Address?

In probabilistic forecasting, neural networks output entire probability distributions rather than single point predictions. While standard practice typically assumes a multivariate Gaussian distribution trained via negative log-likelihood (the log-score), the log-score has a glaring flaw: its quadratic penalty makes it excessively sensitive to outliers and heavy-tailed anomalies. A few extreme events can distort training, causing models to predict unrealistically wide uncertainty intervals.

The Energy Score (ES) is a widely known robust alternative, but it generally lacks a closed-form analytical formula for multivariate distributions. In practice, evaluating the Energy Score requires drawing Monte Carlo samples at every optimization step. This introduces high-variance gradient noise, severely slows down backpropagation (scaling quadratically with sample size), and is known to be relatively insensitive to errors in the correlation structure.

Methods

The authors introduce MVG-CRPS (Multivariate Gaussian CRPS), which bypasses sampling altogether:

  1. PCA Whitening Transformation: Given a predictive Gaussian distribution $\mathcal{N}(\boldsymbol{\mu}_t, \boldsymbol{\Sigma}_t)$ and observation $\mathbf{z}_t$, the covariance is decomposed using singular value decomposition (SVD): $\boldsymbol{\Sigma}_t = \mathbf{U}_t \mathbf{S}_t \mathbf{U}_t^\top$, where $\mathbf{S}_t = \text{diag}(\boldsymbol{\lambda}_t)$.
  2. Orthogonal Decorrelation: Residuals are transformed into uncorrelated variables $\mathbf{v}_t = \mathbf{U}_t^\top (\mathbf{z}_t - \boldsymbol{\mu}_t)$ and standardized to $\mathbf{w}_t = \mathbf{S}_t^{-1/2} \mathbf{v}_t \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$.
  3. Closed-Form Scoring: The multivariate loss is evaluated as the sum of analytical univariate Gaussian CRPS terms along the principal axes: $$\text{MCRPS}(\Phi_N(\boldsymbol{\mu}_t, \boldsymbol{\Sigma}_t), \mathbf{z}_t) = \sum_{i=1}^N \sqrt{\lambda_{i,t}} \, \text{CRPS}(\Phi, w_{i,t})$$
  4. Theoretical Rigor: The paper proves that MVG-CRPS is strictly proper under the Gaussian family using the Schur-Horn majorization theorem and Karamata's inequality.
  5. Practical Parameterization: To maintain stability and prevent eigenvalues from collapsing to zero, the authors parameterize the covariance using a low-rank-plus-diagonal structure ($\boldsymbol{\Sigma}_t = \mathbf{L}_t \mathbf{L}_t^\top + \text{diag}(\mathbf{d}_t)$) with bounded minimum diagonal values, trained over mini-batch subsets ($B \ll N$).

Datasets

  • Real-World Benchmarks: Evaluated across a diverse suite of 13+ standard time series datasets from GluonTS, spanning hourly, daily, weekly, monthly, and yearly granularities:
  • Energy & Traffic: electricity, elec_au, elec_weekly, elec_hourly, traffic, taxi_30min, pedestrian, uber_hourly
  • Finance & Economics: exchange_rate, cif_2016, tourism
  • General & Web: kdd_cup, nn5_daily, saugeenday, sunspot, m1_yearly, m3_yearly, m4_hourly, covid, wiki
  • Synthetic Data: Bivariate Gaussian benchmarks with controlled shifts in mean ($\mu$), scale ($\sigma$), and correlation ($\rho$), alongside controlled outlier injection ($\pm 5\sigma$ perturbation).

Key Insights and Ideas

  • Whitening Makes Multivariate CRPS Analytical: Instead of computing intractable high-dimensional integrals, rotating the space along the eigenvectors of the predicted covariance decouples the variables into independent standard normals, where univariate analytical CRPS formulas apply directly.
  • Sensitivity to Correlation Restored: While the Energy Score often struggles to distinguish misspecifications in correlation, the whitening step in MVG-CRPS explicitly depends on the full covariance matrix $\boldsymbol{\Sigma}_t$. Parameter sensitivity tests show it detects correlation errors as reliably as the log-score.
  • Strict Properness Holds via Majorization: Summing arbitrary marginals usually fails to yield a proper multivariate scoring rule. The authors show that because $f(x) = \sqrt{x}$ is concave, Karamata’s majorization inequality ensures that expected scores are uniquely minimized when both $\boldsymbol{\mu}_q = \boldsymbol{\mu}_p$ and $\boldsymbol{\Sigma}_q = \boldsymbol{\Sigma}_p$.

Key Results

  • Predictive Performance: MVG-CRPS achieved the best average rank across datasets compared to the log-score and Energy Score on both multivariate autoregressive (GPVar and Transformer) and univariate Seq2Seq (N-HiTS) architectures. For example, in GPVar evaluation on $\text{CRPS}_{\text{sum}}$, MVG-CRPS achieved an average rank of 1.46, beating ES (1.92) and the log-score (2.62).
  • Massive Speedup over Energy Score: On GPVar, training with MVG-CRPS was 10× to 50× faster than ES (e.g., ~23 minutes on electricity compared to ~782 minutes for ES; ~29 minutes on elec_au vs. ~717 minutes for ES), with per-epoch runtimes matching the standard log-score.
  • Outlier Robustness: In synthetic noise injection tests, models trained with the log-score deteriorated sharply as noise levels grew, whereas MVG-CRPS maintained stable parameter recovery and produced sharp, well-calibrated confidence intervals.

Limitations and Future Work

  • Parametric Restriction: The analytical derivation relies fundamentally on the multivariate Gaussian assumption (or models using Gaussian copulas).
  • Cubic Complexity of SVD: Backpropagation through SVD has an $O(B^3)$ compute and memory footprint with respect to the evaluated dimension $B$, limiting applicability when evaluating very large joint dimensions simultaneously without sub-sampling.
  • Future Directions: Extending the framework to non-Gaussian distributions via copula transformations, and leveraging isotropic noise parameterizations ($\boldsymbol{\Sigma} = \mathbf{L}\mathbf{L}^\top + \sigma^2 \mathbf{I}$) to accelerate SVD computations for higher dimensions.

Summary updated

Abstract

Multivariate probabilistic forecasting typically leverages neural network-based distributional regression, often employing Gaussian assumptions to simplify computation. While the standard negative log-likelihood provides analytical convenience, its sensitivity to outliers can severely degrade forecasting accuracy. Conversely, robust alternatives like the Energy Score, although less sensitive to extreme values, rely heavily on computationally expensive sampling approximations, limiting scalability in neural network training. To bridge this gap, we introduce the MVG-CRPS, a novel, strictly proper scoring rule for multivariate Gaussian distributions that maintains robustness to outliers while providing a closed-form expression, enabling efficient training and evaluation. Our approach leverages a whitening transformation, decorrelating multivariate outputs and reducing the multivariate scoring task to tractable univariate CRPS computations. Experiments on real-world datasets for both multivariate autoregressive and univariate sequence-to-sequence (Seq2Seq) forecasting tasks demonstrate that MVG-CRPS enhances robustness and predictive performance.
BibTeX
@misc{zheng2024mvg,
  author = {Vincent Zhihao Zheng and Lijun Sun},
  title = {{MVG-CRPS: A Robust Loss Function for Multivariate Probabilistic Forecasting}},
  year = {2024},
  eprint = {2410.09133},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.09133},
}
Figure