Abstract
Estimating quantiles of an outcome conditional on covariates is of fundamental interest in statistics with broad application in probabilistic prediction and forecasting. We propose an ensemble method for conditional quantile estimation, Quantile Super Learning, that combines predictions from multiple candidate algorithms based on their empirical performance measured with respect to a cross-validated empirical risk of the quantile loss function. We present theoretical guarantees for both iid and online data scenarios. The performance of our approach for quantile estimation and in forming prediction intervals is tested in simulation studies. Two case studies related to solar energy are used to illustrate Quantile Super Learning: in an iid setting, we predict the physical properties of perovskite materials for photovoltaic cells, and in an online setting we forecast ground solar irradiance based on output from dynamic weather ensemble models.
Introduction
In traditional statistical modeling and machine learning, the primary focus is often on estimating the conditional mean of a target variable. However, in high-stakes domains such as renewable energy, finance, and healthcare, the "average" outcome provides an incomplete picture. Decision-makers frequently require knowledge of the full conditional distribution—specifically quantiles—to assess risks and manage uncertainty. For instance, in solar energy management, predicting the 95th percentile of irradiance is crucial for grid stability.
While numerous algorithms exist for quantile regression, including Quantile Random Forests and Gradient Boosting Machines, no single model consistently outperforms others across all datasets or time horizons. This paper introduces Quantile Super Learning (QSL), a robust ensemble framework designed to aggregate multiple candidate algorithms to achieve optimal probabilistic forecasts in both static (i.i.d.) and streaming (online) environments.
Methodology
The QSL framework extends the "Super Learner" architecture to the domain of quantile estimation. It operates by maintaining a diverse library of candidate algorithms and determining the optimal combination of these models based on their historical performance.
The Loss Function
Unlike standard regression which minimizes squared error, QSL utilizes the pinball loss (or quantile loss). This asymmetric loss function penalizes predictions based on whether they fall above or below the target, specifically calibrated to a target percentile $\tau$.
Static vs. Online Settings
The researchers developed two distinct approaches for weight optimization:
- I.I.D. Setting: In static environments, QSL employs V-fold cross-validation to estimate the risk of each candidate model on unseen data, ensuring the ensemble weights are not biased by overfitting.
- Online Setting: For streaming time-series data, QSL adopts a "Follow the Leader" strategy. Weights are updated dynamically as new observations arrive, allowing the ensemble to adapt to shifting data distributions.
Theoretical Guarantees
A significant contribution of this work is the derivation of oracle inequalities. These mathematical proofs guarantee that as the sample size increases, the QSL's performance converges to that of the best-performing model (or best linear combination) in the library, even if the optimal model was unknown a priori.
Case Studies and Applications
The efficacy of QSL was evaluated through two primary solar energy applications.
1. Perovskite Material Properties (I.I.D.)
Using a dataset of 1,453 materials from the Materials Project database, the authors applied QSL to predict physical properties essential for photovoltaic cell development, such as formation energy and bandgap. QSL consistently achieved the lowest empirical risk across various quantiles, effectively "hedging" against the weaknesses of individual models like standard Quantile Regression or Gradient Boosting.
2. Solar Irradiance Forecasting (Online)
In the online setting, the researchers forecasted ground solar irradiance using weather ensemble data from the European Centre for Medium-Range Weather Forecasts (ECMWF) across seven US locations. QSL demonstrated superior performance compared to established online aggregation methods, such as Exponentially Weighted Average (EWA) and Bernstein Online Aggregation (BOA).
Key Insights
- Dynamic Weighting: The study revealed that ensemble weights are highly sensitive to the specific quantile being targeted. For example, a model that excels at predicting low-light conditions (10% quantile) may be discarded in favor of another model when predicting peak sun (90% quantile).
- Robustness to Misspecification: QSL acts as a statistical safety net. If a poorly performing model is included in the library, the algorithm learns to assign it near-zero weight, protecting the integrity of the final forecast.
- Calibration of Intervals: By applying QSL to both lower and upper quantiles, the authors constructed prediction intervals. These intervals proved to be more accurately calibrated than those generated by any single constituent model.
Limitations and Future Work
Despite its performance, QSL introduces higher computational overhead due to the need for cross-validation or iterative optimization at each time step. Furthermore, while QSL minimizes quantile loss, it does not inherently guarantee perfect coverage (e.g., a 90% interval covering exactly 90% of points) in small sample sizes. The authors suggest that integrating QSL with Conformal Inference could provide more rigorous coverage guarantees in future iterations.
Conclusion
Quantile Super Learning provides a theoretically grounded and empirically validated method for probabilistic forecasting. By automating the selection and weighting of candidate models, it offers a powerful tool for researchers and practitioners dealing with complex, non-linear, and time-varying data in fields like renewable energy.
Summary updated