Do Time-Series Foundation Models Forecast Index-Futures Returns? A Preregistered Evaluation of Models, Metrics, and Input Representations

Rishabh UpadhyayarXiv 2026 · arXiv

This paper puts the recent wave of Time-Series Foundation Models (TSFMs) to the ultimate test: forecasting the notoriously "unpredictable" intraday returns of liquid index futures. In a rigorous, pre-registered study, the authors find that while these models look impressive on general benchmarks, they provide zero incremental skill over a simple random walk when applied to financial returns.

TL;DR

In a pre-registered blind evaluation of six major time-series foundation models (including Chronos and TimesFM), none were able to outperform a simple "no-change" forecast for intraday index-futures returns. The study reveals that apparent model "skill" often stems from metric artifacts or simple input ingredients rather than actual predictive power.

What Problem Does This Paper Address?

Time-series foundation models have shown remarkable zero-shot capabilities across diverse domains. Recently, finance-specific models like Kronos have reported significant gains in forecasting financial data. However, financial returns are characterized by an extremely low signal-to-noise ratio, making them a "hard mode" for any forecaster.

The authors address a critical question: Is the reported success of TSFMs in finance due to genuine predictive skill, or is it an illusion created by choice of metrics, lack of hard benchmarks (like the random walk), or data leakage? To answer this, they employ a "blind" evaluation protocol—testing models on data they couldn't have seen, using a fixed analysis plan locked before the results were known.

Methods

The study evaluates a panel of five frozen zero-shot models—Chronos-Bolt, Chronos-2, TimesFM 2.5, Moirai 2.0, and TiRex—plus experiments with Kronos.

Key methodological pillars include:

  • Pre-registration: The analysis plan, including models, metrics, and "pass" criteria, was hashed and timestamped before opening the test data to prevent p-hacking.
  • Input Representations: Models were tested with four different inputs: native prices (R0), log returns (R1), volatility-standardized returns (R2), and a volume-based clock (R4).
  • Hard Benchmarks: The primary competitor was the "no-change" forecast ($f=0$), representing the random walk hypothesis. They also used a transformation-aware ridge regression as a control.
  • Planted Signals: To ensure the test could detect a signal if one existed, the authors created synthetic series with known predictability to establish detection limits.

Datasets

The evaluation focused on two highly liquid markets:

  • E-mini Nasdaq-100 (NQ) futures (Primary market)
  • E-mini S&P 500 (ES) futures (Replication market)

The data was split into:

  • Train/Validation: 2018–2024.
  • Blind Set: January 2025 – May 2026 (opened once after freezing the plan).
  • Hold-out ("Ultra-blind"): September – October 2026 (a final descriptive check).

Key Insights and Ideas

  • The RankIC Illusion: One of the most striking findings is that standard metrics like RankIC (rank correlation) can be highly misleading. The authors prove that a trivial "revert-to-mean" rule yields a positive RankIC under a random walk due to the way data is centered within a day. Figure 3 illustrates how a sign-randomized null reproduces ~95% of the observed "skill" in this metric.
  • Ingredient Controls: When a model appeared to beat a volatility benchmark (HAR), the authors found the gain was entirely explained by the Exponential Weighted Moving Average (EWMA) already present in the model's input. The model itself added no incremental value.
  • Magnitude vs. Accuracy: Changing input representations (like log returns vs. native prices) changed the magnitude of the models' forecasts but did not improve their accuracy.
  • Detection Limits: The models could recover strong synthetic signals at a 5-minute horizon, but this ability deteriorated rapidly by 30 minutes. This suggests that for real-world returns, the signal is simply below the models' current resolution.

Key Results

The headline result is a "bounded negative":

  • 0 out of 160 model-representation-horizon-market cells beat the no-change forecast on the blind set.
  • Hansen’s SPA test yielded a p-value of 1.000 for both markets, indicating no evidence of superior predictive ability.
  • As shown in Figure 1, every single Mean Absolute Error (MAE) ratio was above 1.0, meaning the models were consistently less accurate than predicting zero change.
Figure 1: MAE ratio vs no-change, model × input representation × market × horizon, for the blind set (left) and the retrospective hold-out (right). Lower is better; below 1 would beat the random walk. No cell is below 1. ∗partial origin coverage (TimesFM 1,024-step cap on R4); hatched cells were not run (no R4 on the hold-out).
  • Price-level correlation exceeded 0.9996 for all models, but the no-change forecast scored even higher, proving that high correlation in price levels is a meaningless metric for forecasting skill.

Limitations and Future Work

  • Scope: The study is limited to two index futures over a specific timeframe (2023–2026). Results might differ for less liquid assets or different regimes.
  • Frozen Models: The study primarily evaluated "frozen" zero-shot models. While some fine-tuning was tested with Kronos (and also failed), more extensive domain-specific tuning remains a potential avenue.
  • Input Data: The authors noted that without signed trade data (order flow), certain high-frequency signals were not testable.
  • Future Direction: The authors recommend that future financial AI research must use "no-change" as a baseline, calibrate rank metrics against sign-randomized nulls, and explicitly state the detection limits of their study.

Summary updated

Abstract

We ask whether the apparent forecasting skill of frozen time-series foundation models (TSFMs) on intraday index-futures returns is incremental skill. In a pre-registered evaluation on E-mini Nasdaq-100 (NQ) and E-mini S&P 500 (ES) futures, a common panel of five zero-shot models (Chronos-Bolt, Chronos-2, TimesFM 2.5, Moirai 2.0 and TiRex), plus separate experiments with Kronos, is compared with a no-change forecast at 5-120 minutes, using native prices, log returns, volatility-standardised returns and a volume clock as inputs. On a blind set opened once, 0 of 160 model-representation-horizon-market cells beat no-change, and a Hansen SPA test gives p = 1.000 on both markets; a later retrospective hold-out, scored as a descriptive check, gives 0 of 120. Input representations change forecast magnitude but not accuracy, and a transformation-aware ridge regression does as well. Several standard measures nevertheless suggest skill. Price-level correlation exceeds 0.9996, but no-change scores higher. Within-day RankIC across forecast origins rewards a trivial revert-to-mean rule (+0.35 at 30 minutes); we derive the expected within-day covariance under a random walk in closed form, and a sign-randomised null reproduces about 95% of the observed value. One model beats HAR for 5-minute volatility, but not the EWMA its input already contains. On strongly planted synthetic signals the models recover part of the signal at 5 minutes, and recovery deteriorates at 30 minutes. Benchmark-level success of TSFMs should therefore not be assumed to carry over to low-signal domains such as intraday returns.
BibTeX
@misc{upadhyay2026do,
  author = {Rishabh Upadhyay},
  title = {{Do Time-Series Foundation Models Forecast Index-Futures Returns? A Preregistered Evaluation of Models, Metrics, and Input Representations}},
  year = {2026},
  eprint = {2611.12141},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12141},
}
Figure