Regularized Regression for Hierarchical Forecasting Without Unbiasedness Conditions

Souhaib Ben Taieb, Bonsoo Koo

Abstract

Hierarchical time series forecasting requires coherency, where parent node forecasts equal the sum of child node forecasts. Traditional methods like Minimum Trace (MinT) rely on strict unbiasedness assumptions for both base and reconciled forecasts. This paper introduces a regularized Empirical Risk Minimization (ERM) framework that relaxes these constraints, allowing for a bias-variance tradeoff that improves robustness. By penalizing reconciliation weights toward the Bottom-Up solution, the proposed ERMregbu method achieves superior performance in high-dimensional settings and under model misspecification.

Introduction

In many industries, data is naturally organized into hierarchical structures—for example, a retailer tracking sales at the item, store, city, and national levels. A fundamental challenge in forecasting these structures is ensuring coherency: the forecast for a "parent" node (e.g., national sales) must exactly equal the sum of the forecasts for its "children" (e.g., regional sales).

Example of a hierarchical structure where global data is aggregated from national and city levels.

The current state-of-the-art method, MinT (Minimum Trace), reconciles independent "base" forecasts to make them coherent. However, MinT relies on two strict assumptions: that base forecasts are unbiased and that reconciled forecasts must also be unbiased. In practice, base forecasts are almost always biased due to model misspecification or noise. Furthermore, insisting on unbiasedness can lead to high variance, especially in large hierarchies. This paper addresses this gap by allowing for bias in exchange for lower overall forecast error.

Methods

The authors propose an Empirical Risk Minimization (ERM) framework for hierarchical forecasting. Instead of assuming unbiasedness, they treat the reconciliation process as a multi-response regression problem that directly minimizes the Mean Squared Error (MSE).

The hierarchy is defined by a summing matrix $S$, which relates the bottom-level series to the aggregate levels.

Hierarchical notation and the summing matrix S used to enforce coherency.

Key technical contributions include:

  • ERM Framework: The authors derive a closed-form solution for the optimal reconciliation matrix that minimizes empirical risk without the unbiasedness constraint. They demonstrate that MinT is actually a special case of ERM that emerges only under specific unbiasedness conditions.
  • Regularization (ERMreg): To handle "high-dimensional" hierarchies where the number of series $m$ is large relative to the number of observations $T$, they introduce an $L_1$ (LASSO) penalty.
  • Penalization towards Bottom-Up (ERMregbu): Standard LASSO shrinks weights toward zero, which is undesirable for forecasting. Instead, the authors propose penalizing weights toward the "Bottom-Up" (BU) solution. They prove mathematically that the BU approach is the "sparsest" possible coherent solution, making it an ideal robust prior.

Datasets

The framework was evaluated on three distinct data types:

  1. Simulated Data: Generated using ARMA processes to test scenarios with intentional model misspecification.
  2. Road Traffic (PEMS-SF): Occupancy rates of 963 car lanes in the San Francisco Bay Area, aggregated into a hierarchy of regions and sensors.
  3. Wikipedia Webpage Views: Daily views for 145,000 articles, aggregated by agent type, access type, and country.

Key Results

  • Robustness to Misspecification: When the base forecasting model was misspecified (e.g., using Exponential Smoothing on ARMA data), ERMreg significantly outperformed MinT by effectively managing the bias-variance tradeoff.
  • High-Dimensional Success: In large hierarchies like the Wikipedia dataset, unregularized methods suffered from error accumulation. The regularized ERMregbu consistently provided the most accurate results.
  • Sparsity and Interpretability: Visualization of weight matrices showed that ERMregbu produced much sparser, more interpretable reconciliation weights compared to the dense, noisy matrices produced by non-regularized methods.

Conclusion and Future Work

This paper provides a strong theoretical and empirical foundation for hierarchical forecasting without unbiasedness constraints. By treating Bottom-Up as a prior for regularization, the ERMregbu method offers a scalable solution for complex, high-dimensional hierarchies.

Future research directions include addressing computational complexity for even larger hierarchies through parallelization, extending the framework to probabilistic forecasting, and developing new regularization techniques that explicitly exploit the tree structure of the hierarchy.

Summary updated

Abstract

Forecasting a large set of time series with hierarchical aggregation constraints is a central problem for many organizations. However, it is particularly challenging to forecast these hierarchical structures. In fact, it requires not only good forecast accuracy at each level of the hierarchy, but also the coherency between different levels, i.e. the forecasts should satisfy the hierarchical aggregation constraints. Given some incoherent base forecasts, the state-of-the-art methods compute revised forecasts based on forecast combination which ensures that the aggregation constraints are satisfied. However, these methods assume the base forecasts are unbiased and constrain the revised forecasts to be also unbiased. We propose a new forecasting method which relaxes these unbiasedness conditions, and seeks the revised forecasts with the best tradeoff between bias and forecast variance. We also present a regularization method which allows us to deal with high-dimensional hierarchies, and provide its theoretical justification. Finally, we compare the proposed method with the state-of-the-art methods both theoretically and empirically. The results on both simulated and real-world data indicate that our methods provide competitive results compared to the state-of-the-art methods.
BibTeX
@inproceedings{taieb2019regularized,
  author = {Souhaib Ben Taieb and Bonsoo Koo},
  title = {{Regularized Regression for Hierarchical Forecasting Without Unbiasedness Conditions}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2019},
  doi = {10.1145/3292500.3330976},
}
Figure