Abstract
Deep probabilistic time series forecasting has gained attention for its ability to provide nonlinear approximation and valuable uncertainty quantification for decision-making. However, existing models often oversimplify the problem by assuming a time-independent error process and overlooking serial correlation. To overcome this limitation, we propose an innovative training method that incorporates error autocorrelation to enhance probabilistic forecasting accuracy. Our method constructs a mini-batch as a collection of D consecutive time series segments for model training. It explicitly learns a time-varying covariance matrix over each mini-batch, encoding error correlation among adjacent time steps. The learned covariance matrix can be used to improve prediction accuracy and enhance uncertainty quantification. We evaluate our method on two different neural forecasting models and multiple public datasets. Experimental results confirm the effectiveness of the proposed approach in improving the performance of both models across a range of datasets, resulting in notable improvements in predictive accuracy.
Introduction
In the field of time series forecasting, probabilistic models have become essential for quantifying uncertainty. However, many state-of-the-art deep learning models rely on a simplifying assumption: that the errors made at each time step are independent and identically distributed (i.i.d.) "white noise." In practice, time series residuals often exhibit significant autocorrelation. If a model overestimates a value at one time step, it is likely to do so in the next. Ignoring this serial correlation leads to miscalibrated uncertainty intervals and discards valuable information present in the residuals.
Methodology: The "Better Batch" Framework
To address these limitations, we introduce a training method that explicitly models error autocorrelation through two primary innovations.
1. Consecutive Mini-Batch Structure
Standard training treats time windows as independent samples. In contrast, our "Better Batch" approach groups $D$ consecutive segments into a single mini-batch. This structure allows the model to capture how errors evolve over a sequence of adjacent time steps during the training process.
2. Dynamic Covariance Learning and Calibration
We replace the standard univariate Gaussian likelihood with a multivariate Gaussian likelihood. The correlation matrix ($C_t$) is parameterized as a weighted sum of several base kernels (e.g., Squared-Exponential kernels with different lengthscales). A neural network dynamically predicts these weights based on the model's hidden state, allowing the error structure to adapt to changing temporal contexts.
During inference, the model uses the learned correlation matrix to perform real-time calibration. By applying conditional Gaussian distributions, the model adjusts the mean and variance of future predictions based on the errors observed in the most recent data points.
Experimental Results
We evaluated the "Better Batch" method on nine real-world datasets from the GluonTS library using two base architectures: DeepAR (RNN-based) and a GPT-style Transformer.
Accuracy and Uncertainty Improvements
The method achieved significant improvements in the Continuous Ranked Probability Score (CRPS), a standard metric for probabilistic forecasts. Specifically, we observed an average relative improvement of 8.80% for DeepAR and 9.76% for the Transformer. The most notable gains occurred in the distribution tails; for DeepAR, the 0.9-risk (measuring upper uncertainty bound accuracy) improved by 12.36%.
Conclusion
The "Better Batch" approach demonstrates that modeling the relationship between errors—rather than treating them as random noise—significantly enhances both predictive accuracy and uncertainty quantification. By leveraging the structural information within residuals, models can provide more reliable forecasts for complex, real-world time series. Future work will explore extending this framework to multivariate settings and incorporating non-Gaussian error distributions.
Summary updated