ReModels: Quantile Regression Averaging models

Grzegorz Zakrzewski, Kacper Skonieczka, Mikolaj Malkinski, Jacek Mandziuk

TL;DR

The ReModels Python package provides the first unified, open-source implementation of Quantile Regression Averaging (QRA) and its many variants. By streamlining the entire pipeline—from data acquisition via the ENTSO-E API to advanced probabilistic evaluation—it aims to solve the reproducibility crisis in electricity price forecasting.

What Problem Does This Paper Address?

In the volatile world of electricity markets, a single-point forecast (e.g., "the price will be $50/MWh") is rarely enough for effective risk management. Decision-makers need probabilistic forecasts—prediction intervals that account for uncertainty and potential price spikes.

While Quantile Regression Averaging (QRA) has emerged as the "gold standard" for this task (notably winning the GEFCom2014 competition), the field suffers from a lack of transparency. Researchers often develop custom modifications of QRA but rarely share their code or exact data-processing steps. This makes it nearly impossible to compare different models fairly or build upon previous work. ReModels bridges this gap by providing a standardized, modular toolkit for the research community.

Methods

The core of the package is the implementation of nine distinct QRA variants. QRA works by taking point forecasts from multiple models and using them as independent variables in a quantile regression to estimate specific percentiles of the future price distribution.

The package implements:

  • Standard QRA: The base model.
  • Regularized Variants: LQRA (using L1/LASSO penalty) to handle high-dimensional inputs.
  • Factor-based Variants: FQRA and sFQRA, which use Principal Component Analysis (PCA) to reduce the dimensionality of input forecasts.
  • Smoothed Variants: SQRA and SQRM, which incorporate kernel density estimation to make the cost function convex and easier to optimize.
  • Averaged Variants: QRM and FQRM, which average point forecasts before applying regression.

Beyond the models, the package introduces a modular architecture:

  1. Data Acquisition: Automated fetching of market data.
  2. Data Preparation: Implementation of eight Variance Stabilizing Transformations (VSTs) (like Box-Cox, Arcsinh, and Logit) to handle the extreme outliers common in energy prices.
  3. Point Forecasting: A wrapper compatible with scikit-learn pipelines.
  4. Evaluation: Specialized metrics for probabilistic accuracy, such as the Aggregate Pinball Score (APS) and Average Empirical Coverage (AEC).

Datasets

The paper demonstrates the package using data from the German electricity market.

  • Source: The ENTSO-E Transparency Platform.
  • Content: Historical day-ahead electricity prices and load forecasts from January 5, 2015, to January 1, 2017.
  • Accessibility: The package includes a dedicated EntsoeApi class to allow users to easily replicate these datasets or download data for other European regions.

Key Insights and Ideas

  • The "Pipeline" Philosophy: The authors treat forecasting as a holistic process. By integrating VSTs directly into the workflow, they acknowledge that how you transform your data is often as important as the model itself.
  • Standardization as a Catalyst: By providing a PointModel class that follows the scikit-learn API, the authors make it easy for engineers to swap out simple linear regressions for complex machine learning models while keeping the probabilistic "averaging" layer intact.
  • Handling the "Spike": Electricity prices are notorious for "spikes" (extreme outliers). The inclusion of eight different VSTs allows researchers to systematically test which transformation best tames these spikes for a specific market.

Key Results

The authors provide an illustrative example comparing four QRA variants on the German dataset:

  • Coverage Accuracy: For a 50% expected coverage interval, the SQRM (Smoothed Quantile Regression Averaging with point forecast averaging) achieved 52.30% actual coverage, performing closer to the ideal than the base QRA (45.17%).
  • High-Confidence Intervals: At the 90% coverage level, all tested variants performed well, with SQRM reaching 88.87%, indicating high reliability for extreme price bounds.
  • Calibration Sensitivity: The results (Table 5) show that the choice of the "calibration window" (how much historical data the model sees) significantly impacts error metrics like MAPE, highlighting the need for the package's automated testing tools.

Limitations and Future Work

  • Linearity: The current QRA variants in the package are primarily linear. While effective, they may miss complex non-linear relationships between input forecasts.
  • Future Models: The authors plan to expand the library to include "distant cousins" of QRA, such as Quantile Regression Forests and Quantile Regression Neural Networks, which can handle non-linear dynamics.
  • Domain Specificity: While currently optimized for energy prices, the authors note that the package could be extended to other fields like wind power or nodal voltage forecasting.

Summary updated

Abstract

Electricity price forecasts are essential for making informed business decisions within the electricity markets. Probabilistic forecasts, which provide a range of possible future prices rather than a single estimate, are particularly valuable for capturing market uncertainties. The Quantile Regression Averaging (QRA) method is a leading approach to generating these probabilistic forecasts. In this paper, we introduce ReModels, a comprehensive Python package that implements QRA and its various modifications from recent literature. This package not only offers tools for QRA but also includes features for data acquisition, preparation, and variance stabilizing transformations (VSTs). To the best of our knowledge, there is no publicly available implementation of QRA and its variants. Our package aims to fill this gap, providing researchers and practitioners with the tools to generate accurate and reliable probabilistic forecasts in the field of electricity price forecasting.
BibTeX
@inproceedings{zakrzewski2024remodels,
  author = {Grzegorz Zakrzewski and Kacper Skonieczka and Mikolaj Malkinski and Jacek Mandziuk},
  title = {{ReModels: Quantile Regression Averaging models}},
  booktitle = {SoftwareX},
  year = {2024},
  url = {https://openreview.net/forum?id=z8YKtHD3pi},
}

Loading related content...

Figure