In many real-world forecasting applications, data is naturally structured into a hierarchy. A retail chain tracks sales from individual items up to stores, regions, and the entire country. An electric utility monitors power consumption from individual smart meters up to neighborhoods, substations, and the entire grid.
When generating forecasts for these systems, a naive approach is to train independent models for each node in the hierarchy. However, this creates a fundamental accounting problem: the independent forecasts will almost never add up correctly. As noted by Tsiourvas et al. in their work on optimal projection learning, if a model predicts 100 units sold nationwide, but the constituent regional models sum to 90, the forecast is logically broken. This lack of mathematical consistency—known as incoherence—undermines downstream decision-making and resource planning.
Forecast reconciliation is the process of adjusting these independent "base" forecasts so that they perfectly satisfy the structural summation constraints of the hierarchy.
The Geometry of Reconciliation
To understand why reconciliation works, it helps to view the problem geometrically.
Let $y_b$ be a vector of the finest-grained "bottom-level" series (e.g., individual store sales). We can represent the entire hierarchy $y$ by multiplying $y_b$ by a structural summing matrix $S$:
$$y = S y_b$$
Because of this linear relationship, true future values must lie within a specific linear subspace defined by $S$. When we generate independent base forecasts $\hat{y}$ across different levels of the hierarchy, these predictions typically fall outside this valid subspace due to noise and model misspecification.
Reconciliation acts as a geometric projection. We take the incoherent base forecasts $\hat{y}$ and map them back onto the coherent subspace using a projection matrix $P$:
$$\tilde{y} = S P \hat{y}$$
The matrix $P$ determines how errors are redistributed across the hierarchy. Early classical methods relied on fixed, post-processing projections (such as Ordinary Least Squares or Weighted Least Squares based on historical residuals). However, modern approaches treat this projection not as a static afterthought, but as an optimization problem that can be learned directly from data.
Hierarchy as a Regularizer
It is tempting to view forecast reconciliation merely as a cleaning step to fix an accounting bug. However, the core insight of recent research is that enforcing coherence acts as a powerful denoising mechanism that actively improves forecasting accuracy.
Time series data at different scales contains different signal-to-noise ratios:
- Bottom-level series (e.g., hourly demand at a single smart meter) are highly volatile, sparse, and plagued by localized noise.
- Top-level aggregates (e.g., daily demand across an entire city) are smooth, stable, and clearly expose long-term trends and seasonal patterns.
By forcing the bottom-level forecasts to align with the top-level aggregates, we inject the stable, global signal from the top of the hierarchy down into the noisy bottom nodes.
As demonstrated by Deshpande et al. in the context of temporal hierarchies (aligning daily, weekly, and monthly forecasts), this cross-scale consensus acts as a robust regularizer. Similarly, Cini et al. showed in their work on Hierarchical Graph Predictors that embedding a differentiable projection layer directly into a neural network prevents the model from overfitting to localized noise in individual data streams, leading to more stable predictions across all resolutions.
End-to-End Differentiable Reconciliation
Traditional reconciliation frameworks operate strictly in a two-step pipeline: first generate base forecasts, then project them. The drawback is that the base forecasting models are completely blind to the aggregation constraints during training.
To bridge this gap, researchers have moved toward end-to-end learning, where the reconciliation step is made fully differentiable and integrated directly into the neural network architecture.
[ Historical Data ] ──> [ Neural Network ] ──> [ Base Forecasts (Incoherent) ]
│
▼
[ Final Coherent Loss ] <── [ Differentiable Projection Layer (S * P) ]
By placing the loss function after the projection layer, the network minimizes the error of the final, coherent predictions. This allows the model to learn internal representations that are inherently optimized for the hierarchy.
Several architectures have successfully operationalized this intuition:
- Empirical Risk Minimization: Ben Taieb et al. reframed the problem as a regularized empirical risk minimization task, abandoning traditional unbiasedness conditions to achieve robustness in high-dimensional settings where data is scarce.
- Differentiable Layers: Rangapuram et al. introduced a differentiable projection layer for temporal hierarchies, allowing joint optimization across multiple time granularities simultaneously.
- Convex Optimization Layers: In domains with strict physical constraints, such as electric vehicle charging networks, Zheng et al. utilized Differentiable Convex Optimization Layers (DCL) to learn optimal reconciliation weights while preventing non-physical anomalies like negative demand predictions.
- Task-Based Optimization: Zhou et al. extended this to business utility with SLOTH, a framework that optimizes the reconciliation layer against downstream decision costs rather than just standard statistical errors.
The Frontier: Probabilistic Coherence
While reconciling point forecasts (mean values) reduces to linear algebra, real-world decision-making requires understanding uncertainty. Moving from point estimates to probabilistic forecast reconciliation introduces a much steeper challenge: ensuring that the entire predictive distributions sum up coherently.
If the bottom-level forecasts are represented as probability distributions, their sum must mathematically equal the joint distribution of the aggregate level. This requires modeling complex, non-Gaussian dependencies across the branches of the hierarchy.
To solve this, Ben Taieb et al. proposed using recursive copula-based aggregation to capture joint dependencies across series while maintaining sparse mean reconciliation. When full distributions are too computationally expensive, quantile forecasting serves as a powerful alternative. Han et al. developed SHARQ (Simultaneous HierArchically Reconciled Quantile Regression), which integrates a regularized penalty directly into the quantile loss function. This enforces coherence across multiple quantiles simultaneously during training, bypassing expensive post-processing and providing computationally efficient, distribution-free uncertainty bounds.
Key Foundational Work
Game-theoretically Optimal Reconciliation of Contemporaneous Hierarchical Time Series Forecasts (2013) — contributed 2 concept elements to this direction.
Sparse and Smooth Adjustments for Coherent Forecasts in Temporal Aggregation of Time Series (2016) — contributed 2 concept elements to this direction.