Federated Hierarchical Stacking of Electricity Price Forecasts

Electricity prices are hard to forecast in a lopsided way. On most hours, simple statistical models such as exponential smoothing or ARIMA are hard to beat. On a few hours, such as heatwaves, scarcity events, or supply shocks, prices spike, those models fail badly, and the hours they miss are the ones that cost money. Stacking offers a direct fix. Stacking is an ensemble method in which a second-level "meta-learner" is trained to decide how much to trust each of several forecasters. If the meta-learner's weights depend on current conditions, it can lean on the cheap statistical model in calm regimes and shift weight to a nonlinear model when volatility rises. The difficulty is that every market or grid operator sees only a handful of spike episodes in its own history. That is far too few to learn a reliable rule for when to switch, and data-sharing constraints usually prevent pooling raw histories across markets.

We propose that operators should federate the combination rule and nothing else. Each participant keeps its own forecasting models and data, which may be of any architecture. It shares only the few dozen parameters of a meta-learner that maps regime indicators to combination weights. Those shared parameters are partially pooled toward a common prior: a market with little evidence about spike regimes borrows the others' switching rule, and a market with plenty of evidence keeps its own. The claim is that this recovers most of the accuracy gain that input-dependent stacking offers on spike hours and in markets with short histories. It does so at a communication cost several orders of magnitude below parameter-sharing federated learning, and it needs neither a shared evaluation dataset nor identical models across participants.

Core Claim

Three bodies of prior work each supply part of the idea but not the whole.

The forecasting literature motivates the design by showing that no single model family wins everywhere. Linear baselines smooth over the nonlinear temperature thresholds that drive peak demand (Energy consumption prediction using people dynamics derived from cellular network data). Yet ARIMA still beats GPT-4o and Gemini in mean absolute error on many half-hourly price cases (NSW-EPNews). Transformers lose badly on the volatile 39-country Real-E grid, where their MAE rises 85.4% (Real-E). In our design, classical models are therefore base forecasters to be kept, not strawmen to be beaten.

Stacking supplies the method. Bayesian hierarchical stacking generalizes constant combination weights to input-dependent weights $w_k(x)$ with partial pooling across groups. That paper leaves three things open, and our proposal addresses each one. It is centralized, it relies on MCMC over leave-one-out log densities, and it is derived only for the log score rather than the pinball loss used for energy quantile forecasts.

Federated learning supplies the constraint. It also supplies the two closest existing systems, and our design differs from both. FedStack lets clients run different architectures and sends their predictions on shared benchmark sequences to a server-side meta-learner. It does this for activity classification, and it assumes an aligned shared evaluation set, which does not exist across separate power markets. EcoStack-Pro stacks gradient-boosted trees locally and has the server weight whole clients by their validation error and generalization gap. It works on static annual tabular data and uses a single weight per client, not a rule that depends on conditions.

Our design sends no predictions and no base-model weights. The object that is federated is the conditional switching rule itself. It inverts the logic of personalization layers, which share a recurrent representation and keep the output head local. We keep the entire forecasting stack local and share only the thin decision layer on top of it.

Research Program

Each client $c$ is a bidding zone, a country, or, in the load variant, a building. It maintains up to $K$ base forecasters from agreed "slots." The slots are a seasonal-naive or exponential-smoothing model, a gradient-boosted tree on exogenous covariates, a small neural network such as N-HITS or an LSTM, and optionally a frozen foundation model such as Chronos. A client may leave any slot empty. The softmax is then taken over only the slots it has, which accommodates differences in hardware and in-house models.

The meta-learner produces client-specific weights over these forecasters:

$$ w_{c,k}(x_t) = \mathrm{softmax}_k\big(\alpha_{c,k} + \beta_{c,k}^\top x_t\big) $$

Here $x_t$ is a short vector of regime indicators, all computable causally at forecast origin $t$: a rolling price volatility, a residual-load or temperature proxy, the hour of day, a day-type flag, and the recent spread between base forecasts. We write $\theta_c = (\alpha_c, \beta_c)$ for client $c$'s meta-parameters.

The combined quantile forecast at level $q$ is the weighted average of the base models' $q$-quantiles. Clients jointly minimize the following objective:

$$ \sum_c \Big[\sum_{t,q} \rho_q\big(y_{c,t} - \textstyle\sum_k w_{c,k}(x_t)\,\hat y^{(q)}_{c,k,t}\big) + \tfrac{1}{2\tau^2}\lVert \theta_c - \mu\rVert^2\Big] $$

In this objective, $\rho_q$ is the pinball loss at quantile level $q$, $y_{c,t}$ is the realized value, $\hat y^{(q)}_{c,k,t}$ is base model $k$'s quantile forecast, $\mu$ is the shared prior mean, and $\tau$ controls how strongly clients are pulled toward it.

Optimization alternates between clients and server. Clients take gradient steps on their own term. The server sets $\mu$ to the average of the $\theta_c$ and re-estimates $\tau$ from their spread, which is an empirical-Bayes step. The result is a maximum a posteriori version of hierarchical stacking under pinball loss, with no MCMC. When $\tau \to 0$ the model reduces to one global rule (complete pooling); when $\tau \to \infty$ it reduces to independent local stacking. Estimating $\tau$ is how the data decide between the two.

The second technical piece is leakage-free training data for the meta-learner. Standard stacking trains the meta-learner on "out-of-fold" predictions, meaning predictions each base model makes on data it was not fit on. Ordinary k-fold splits break this for time series because they let a model see the future. Each client instead generates rolling-origin out-of-sample forecasts. Base models are refit on an expanding window at a fixed schedule, for example monthly, and forecast only the following block. The meta-learner sees only these genuinely out-of-sample forecasts, and its own evaluation uses a later holdout period.

We will measure the leakage directly by also running k-fold stacking. We expect it to overweight the most flexible model and to underperform out of sample. This would give the first quantified account of the temporal-leakage open problem in a federated setting.

Communication per round is one vector $\theta_c$ of roughly $K \times 8$ numbers. That is small enough to add differential-privacy noise cheaply. We will report the accuracy cost of that noise.

Baselines

The naive reference points are two. The first is a seasonal-naive forecast, which repeats last week's value for the same hour. The second is the equal-weight average of each client's own base forecasts. The equal-weight average is the real floor: forecast combination often shows that learned weights fail to beat equal weights. If we cannot beat it on overall pinball loss, or at least on spike hours, the proposal fails.

The strongest existing method on the price benchmark is the covariate-guided MoE-Encoder of One-Shot Price Forecasting with Covariate-Guided Experts under Privacy Constraints. It trains a sparse mixture-of-experts module with LoRA on a frozen Chronos model, federated across the same five markets we use. It is the strongest competitor because it also routes on covariates under privacy constraints and reports the lowest Weighted Quantile Loss (WQL) on three of those datasets. Its per-dataset WQL and MASE values are not given in the summary; no reported result was found on Lacuna for MoE-Encoder WQL on electricity-BE/DE/FR/NP/PJM. We will therefore rerun it on our splits, with the same Chronos checkpoint, the same forecast horizons and origins, and a tuning budget matched in GPU-hours to the total cost of our base models plus meta-learner. Beating it means lower WQL on at least four of five markets and lower MAE on the top 5% of price hours, tested with Diebold-Mariano tests across origins.

On the many-client load variant, the strongest federated competitor is DualEncDecoder. It reports SMAPE of 22.94% (with difficulty-aware client sampling, no clustering) and NRMSE of 26.62% (with difficulty-aware sampling and clustering) on 1,410 ASHRAE buildings. We will compare against those figures on a reproduced split. We will also compare against PL-FL, whose MASE numbers are not given in the file.

Four ablations isolate the key ingredient. The first uses no pooling, so each client runs its own input-dependent stacking. The second uses complete pooling, with one global rule. The third uses constant-weight stacking, removing the dependence on $x_t$. The fourth trains on k-fold instead of rolling-origin out-of-fold forecasts. The central comparison is partial pooling against no pooling.

Why not just…?

Why not just let each market do its own stacking? This is the most serious threat. With years of hourly data, a market may have enough evidence that pooling adds nothing on average. We expect pooling to matter only where local evidence is thin: spike regimes, markets that entered recent crisis phases (as the three-phase Spanish segmentation in DROI illustrates), and newly joining clients. We will test these directly by truncating each client's history to 3, 6, and 12 months and by scoring spike hours separately. If partial pooling beats no pooling nowhere, even under truncation, the federation part of the idea is refuted.

Why not just average the forecasts equally? For energy stacking, the only head-to-head evidence in the file is on 52 annual oil observations, where stacking reached 1.65% MAPE against 2.52% for bagging (SMLE). That is too small to trust. We expect equal weights to be near-optimal on calm hours and poor on spikes, where the seasonal-naive slot is systematically wrong. The spike-hour breakdown will show whether condition-dependent weights earn their added variance.

Why not just fine-tune one federated foundation model with a mixture of experts? The MoE-Encoder already routes on covariates. However, its gate is trained end to end with the experts, it lives inside one architecture, and it has no pooling mechanism for rare regimes. Real-E suggests that attention-based models are fragile precisely under high volatility. We will compare fairly by including MoE-Encoder as a slot in our own stack as well as running it standalone. If stacking it with classical models does not improve on it alone, that is a negative result for our proposal.

Proposed Experiments

The primary price testbed has six clients. Five are the Zenodo electricity-BE/DE/FR/NP/PJM datasets used in One-Shot Price Forecasting. The sixth is the half-hourly NSW price series from NSW-EPNews, used without its news text. We forecast day-ahead (24 or 48 steps). Because six groups give a weak estimate of $\tau$, we add a many-client load testbed: ASHRAE with 1,410 buildings, and Real-E country load series where 39 countries are clients. There, partial pooling can be tested with enough groups to matter, and the volatility covariate can include Real-E's temporal-graph-volatility statistic.

Metrics are WQL (pinball loss averaged over quantiles), MASE, MAE on top-5% price hours, 90% interval coverage, and bytes communicated per round. For load, we add SMAPE and NRMSE for comparability with DualEncDecoder.

The decisive checks are three. First, partial pooling should beat no pooling specifically on spike hours and truncated-history clients, and should roughly tie on calm hours. Second, the learned rule should put more weight on the nonlinear slots as volatility rises, and this should hold consistently across clients. Third, k-fold out-of-fold training should measurably hurt out-of-sample WQL relative to rolling-origin training. If the first check fails while the second holds, the result is still useful: local condition-dependent stacking for electricity forecasting would be justified, but federation of the combination rule would not.