Conformal Expert Pools: Training-Free Probabilistic Time-Series Forecasting Beyond Seasonality

Valery ManokhinarXiv 2026 · DOI · arXiv

In the world of time-series forecasting, the prevailing trend is toward increasingly complex "global" models—deep neural networks trained on massive datasets to learn universal patterns. However, a new paper by Valery Manokhin challenges this paradigm, introducing Conformal Expert Pools (CEP). CEP is a training-free, probabilistic forecaster that manages to outperform state-of-the-art learned models while being orders of magnitude faster.

TL;DR

Conformal Expert Pools (CEP) is a training-free probabilistic forecasting method that uses a pool of simple experts and conformal prediction to achieve superior accuracy and better-calibrated prediction intervals than complex neural models like Amazon's DeepNPTS. It requires zero cross-series training, adapts instantly to new data, and runs hundreds of times faster than its competitors.

What Problem Does This Paper Address?

Probabilistic forecasting—providing a range of plausible future values rather than a single number—is critical for decision-making under uncertainty. While deep learning models have made strides here, they are computationally expensive to train and can struggle with "out-of-distribution" series.

A simpler predecessor, Conformal Seasonal Pools (CSP), showed that a training-free approach based on past seasonal values and errors could beat complex models. However, CSP was limited: it only used a single "seasonal-naive" forecast, making it brittle on trending series, short histories, or data with changing volatility. CEP addresses these gaps by evolving from a single forecaster to an adaptive pool of experts.

Methods

CEP is built on two core principles:

  1. Expert Pools: Instead of relying on one forecast method, CEP maintains a pool of simple, classical point forecasters (the "experts"). These experts are not "trained" in the traditional sense; they have fixed settings and are applied directly to the series' history.
  2. Empirical Error Pools (Conformal Prediction): CEP doesn't assume a normal distribution for uncertainty. Instead, it builds its predictive distribution by sampling from the actual signed residuals (forecast errors) these experts made on the series' own past.

Crucially, CEP is training-free. No parameters are learned across different series. It adapts to each series individually by selecting the best-performing experts and error distributions from that series' specific history.

Datasets

The authors evaluated CEP using a massive, diverse benchmark:

  • CSP Benchmark: 232 datasets containing 24,502 forecast windows, covering domains like energy, transport, finance, and weather.
  • fev-bench: 36 additional "fresh" files (1,341 windows) from families like cloud workloads and epidemiology to test generalization outside the primary benchmark.
  • Laplace Subset: 2,534 windows from 228 datasets specifically used to benchmark against the "Laplace" online ensemble forecaster.

Key Insights and Ideas

  • Simplicity Wins: CEP demonstrates that "global" training isn't a prerequisite for high performance. By focusing on the local history of a series, CEP avoids the overhead and potential pitfalls of cross-series learning.
  • Calibration is King: One of the most striking findings is the failure of complex models to provide reliable prediction intervals. While DeepNPTS's 95% intervals only contained the true outcome 57.8% of the time, CEP achieved 92.2% coverage.
  • The "Laplace" Rebuttal: The paper provides a rigorous critique of the Laplace forecaster, which had been claimed to "beat (almost) everything." Manokhin shows that on multi-step horizons, Laplace's performance collapses, producing biased point forecasts and pathologically wide intervals (sometimes 15 orders of magnitude wider than the data).
  • Efficiency as a Feature: Because it avoids training, CEP is incredibly fast. It can forecast the entire M4 competition (100,000 series) in about 66 seconds, whereas a model like Laplace would take over 50 processor-hours.

Key Results

CEP emerged as the strongest method on the benchmark, achieving the best mean rank across all tested probabilistic scores.

  • Accuracy: CEP outperformed CSP on 78% of datasets by CRPS (Continuous Ranked Probability Score) and 84% by MSIS (Mean Scaled Interval Score).
  • vs. Deep Learning: CEP achieved lower CRPS than Amazon's DeepNPTS on 88% of datasets.
  • Speed: CEP has a median forecast time of just 39 milliseconds. It is roughly 271x faster than Laplace and 819x faster than DeepNPTS.
  • Point Forecasts: Even as a point forecaster, CEP is superior, lowering the median MASE (Mean Absolute Scaled Error) from 1.10 (CSP) to 1.01.
Table 1: Full corpus results showing CEP's superior ranking and coverage compared to baselines.
Table 2: Head-to-head "win" rates and score ratios against individual baselines.

Limitations and Future Work

The authors acknowledge that CEP is "deliberately minimal." It does not use covariates (like weather data or holidays) or leverage information from other related series, both of which are standard in many industrial applications.

While CEP provides excellent empirical coverage, it lacks a formal theoretical coverage guarantee for all types of non-stationary temporal dependence. The authors suggest that CEP should now serve as the "mandatory baseline" that any new, complex forecasting model must be required to beat.

Summary updated

Abstract

Conformal Seasonal Pools (CSP) showed that a forecaster with no training at all, built only from past values at the same point in the season and from past forecast errors, significantly outperformed DeepNPTS, Amazon's learned non-parametric forecaster. We propose Conformal Expert Pools (CEP), a training-free successor. CEP replaces the single seasonal-naive forecast by a pool of simple forecasters, builds its predictive distribution from their past forecast errors, and adapts both the pool and the errors to the series being forecast. No model is trained and no parameter is learned across series. On the CSP benchmark of 232 datasets and 24,502 forecast windows, CEP is better than CSP on 78% of datasets by CRPS, on 84% by MSIS and on 83% by Winkler score; on a typical dataset its CRPS is 14% lower and its MSIS 23% lower (both p < 1e-22, paired Wilcoxon test), and coverage of 95% intervals rises from 90.1% to 92.2%. CEP has a lower CRPS than DeepNPTS on 88% of datasets, than NPTS on 89% and than SeasonalNPTS on 83%. The improvement persists on dataset families outside the development set and on 36 additional fev-bench files. We also test the published claims that Laplace, an online ensemble forecaster, "beats (almost) everything" and "beats" CSP. On 228 datasets at multi-step horizons the claims do not hold: CSP outperforms Laplace on 60% of datasets by CRPS and on 64% by interval score, and Laplace ranks sixth of seven methods, with the widest intervals of all (69% wider than CEP's). Its point forecast is biased low and on 54% of datasets is less accurate than seasonal naive, and from a cold start it takes about 7,700 times longer than CSP. CEP is better than Laplace on 82% of datasets by CRPS and on 80% by MSIS, with intervals about 40% narrower, and runs 271 times faster.
BibTeX
@misc{manokhin2026conformal,
  author = {Valery Manokhin},
  title = {{Conformal Expert Pools: Training-Free Probabilistic Time-Series Forecasting Beyond Seasonality}},
  year = {2026},
  eprint = {2611.21267},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21267},
}
Figure