Appendix for Nonparametric Multivariate Probability Density Forecast in Smart Grids With Deep Learning

Zichao Meng, Guo Ye, Wenjun Tang, Hongbin Sun

Abstract

This paper presents a novel nonparametric multivariate density forecasting model designed for smart grid applications. Unlike traditional methods that rely on parametric assumptions or copula functions, the proposed framework utilizes a specialized Joint Deep Architecture Network (JDAN) to approximate joint cumulative distribution functions (JCDFs). By leveraging the universal approximation capabilities of neural networks while enforcing specific mathematical constraints, the model captures both marginal distributions and complex inter-variable correlations. Numerical evaluations on wind speed, wind power, and electricity load data demonstrate that the JDAN outperforms existing models in terms of distributional accuracy, prediction interval precision, and correlation modeling.

Introduction

In the management of modern smart grids, the ability to forecast multiple correlated variables is essential for maintaining stability and optimizing resource allocation. For instance, grid operators must understand the joint behavior of wind speed and power output across different geographical sites, or the relationship between total system load and pricing.

Traditional forecasting methods often focus on point estimates or individual marginal distributions. When multivariate relationships are considered, researchers typically rely on "copulas" or specific parametric assumptions that pre-define the mathematical shape of the relationship. However, if these assumptions do not align with the actual data distribution, the resulting forecasts can be highly inaccurate. This paper addresses the need for a nonparametric approach that can learn the underlying joint distribution directly from data without prior hypotheses.

Methodology: The Joint Deep Architecture Network (JDAN)

The core contribution of this research is the development of the Joint Deep Architecture Network (JDAN). Creating a neural network capable of outputting a valid Joint Cumulative Distribution Function (JCDF) requires satisfying rigorous mathematical conditions: the function must be multivariate monotonically non-decreasing, and its higher-order derivatives must ensure that the resulting probability density is never negative.

The Failure of Standard Architectures

A significant finding in this work is that standard deep learning architectures are often unsuitable for joint density forecasting. While positive-weighted neural networks (such as the DAN-NFN model) can ensure monotonicity, they often fail to maintain the required higher-order derivative properties.

For example, the widely used ReLU activation function has a non-negative first derivative, but its second derivative is zero almost everywhere. In a multivariate context, this means a ReLU-based network cannot effectively learn the correlations between variables because the gradients required to describe those interactions vanish. Similarly, functions like Sigmoid or Tanh can produce negative higher-order derivatives, leading to mathematically invalid (negative) probability densities.

The JDAN Solution

To overcome these limitations, the authors designed the JDAN architecture to be "valid by design." Using mathematical induction, the paper proves that the JDAN mapping maintains non-negativity for joint density across any number of variables ($D \geq 2$). This is achieved through a specific arrangement of positive weights and carefully selected activation functions that satisfy the necessary derivative conditions, ensuring the model remains a mathematically sound representation of a JCDF while retaining the universal approximation power of deep learning.

Data and Validation

The model was validated using several high-resolution datasets relevant to smart grid operations:

  • Wind Speed and Power: Data sourced from the West Texas Mesonet, providing a benchmark for renewable energy fluctuations.
  • Electricity Load and Price: Aggregated data from the Australian Energy Market Operator (AEMO) to test the model's ability to handle demand-side variables.
  • Renewable Energy Statistics: General benchmarks from international power sector analysts were used to calibrate the forecasting scenarios.

Key Results

The JDAN method was compared against current state-of-the-art multivariate density forecast models. The results highlight three primary advantages:

  1. Accordance with Reality: The predicted joint distributions more closely mirrored the empirical distributions of the observed data compared to parametric alternatives.
  2. Prediction Interval Width: The model generated narrower, more precise confidence intervals without sacrificing reliability, providing more actionable information for grid operators.
  3. Correlation Accuracy: The JDAN successfully captured the evolving correlations between different random variables, a task where traditional marginal-focused models typically struggle.

Discussion and Future Work

While the JDAN represents a significant step forward, the research identifies several ongoing challenges. The choice of activation function involves a trade-off; while functions like the exponential function ($e^x$) satisfy derivative requirements, they can introduce training instabilities such as vanishing or exploding gradients. Furthermore, as the dimensionality ($D$) of the forecasting target increases, the computational complexity of calculating higher-order derivatives grows, suggesting a need for further optimization in high-dimensional smart grid environments.

In conclusion, the JDAN framework provides a robust, nonparametric solution for multivariate forecasting, offering a mathematically rigorous way to model the complex dependencies inherent in modern energy systems.

Summary updated

Abstract

This paper proposes a nonparametric multivariate density forecast model based on deep learning. It not only offers the whole marginal distribution of each random variable in forecasting targets, but also reveals the future correlation between them. Differing from existing multivariate density forecast models, the proposed method requires no a priori hypotheses on the forecasted joint probability distribution of forecasting targets. In addition, based on the universal approximation capability of neural networks, the real joint cumulative distribution functions of forecasting targets are well-approximated by a special positive-weighted deep neural network in the proposed method. Numerical tests from different scenarios were implemented under a comprehensive verification framework for evaluation, including the very short-term forecast of the wind speed, wind power, and the day-ahead forecast of the aggregated electricity load. Testing results corroborate the superiority of the proposed method over current multivariate density forecast models considering the accordance with reality, prediction interval width, and correlations between different random variables.
BibTeX
@misc{meng2022appendix,
  author = {Zichao Meng and Guo Ye and Wenjun Tang and Hongbin Sun},
  title = {{Appendix for Nonparametric Multivariate Probability Density Forecast in Smart Grids With Deep Learning}},
  year = {2022},
  eprint = {2210.15463},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2210.15463},
}
Figure