TL;DR
Chronos-2 is a 120M-parameter foundation model that brings zero-shot multivariate forecasting to time series. By combining a novel group-attention mechanism with a massive synthetic multivariate training corpus, it can infer complex interactions between variables and covariates directly from context, outperforming specialized models while being up to 1000x faster end-to-end.
What Problem Does This Paper Address?
Time series forecasting has long been dominated by two extremes: simple statistical models that lack capacity, and complex machine learning models that require expensive, task-specific training and tuning for every new dataset. While recent Time Series Foundation Models (TSFMs) have shown promise in "zero-shot" forecasting (predicting on unseen data without retraining), most are strictly univariate—they treat every variable in isolation.
Real-world systems, however, are inherently coupled. Predicting an epidemic requires looking at interconnected populations; forecasting energy demand depends on weather covariates; and modeling an economy involves tracking dozens of co-evolving indicators. The challenge is building a single, generalist model capable of capturing these multivariate dependencies and heterogeneous covariates across diverse scientific and industrial domains without needing to be retrained for each one.
Methods
Chronos-2 is an encoder-only Transformer (based on the T5 architecture) that treats forecasting as a sequence-to-sequence task. Its core innovation lies in how it processes multivariate data:
- Group Attention: The model alternates between standard temporal attention (looking at patterns within a single series over time) and a new group attention layer. Group attention aggregates information across different related series or covariates at the same time index. This allows the model to perform In-Context Learning (ICL), inferring relationships between variables (e.g., how temperature drives energy load) purely from the historical data provided at inference time.
- Tokenization & Patching: To handle diverse scales, the model uses a robust scaling technique (standardization followed by a
sinh^-1transformation). It then breaks the series into non-overlapping patches of 16 time steps. Unlike autoregressive models that generate one step at a time, Chronos-2 decodes multiple forecast patches in a single forward pass, significantly boosting speed. - Probabilistic Quantile Head: Instead of a single point estimate, the model outputs 21 quantiles. This allows it to represent complex, multi-modal uncertainty distributions without assuming the data follows a specific parametric form like a Normal distribution.
Datasets
The authors argue that real-world multivariate data is often too scarce or biased for pretraining. Instead, they developed a massive synthetic pipeline:
- Training: A mix of 50M real univariate series and a 12M-series synthetic multivariate corpus. The synthetic data uses generators like
KernelSynthandVARto simulate diverse instantaneous and sequential interactions (see Figure 2). - Evaluation: The primary benchmark is fev-bench (100 tasks, 68 of which are multivariate). They also tested on Chronos-Bench-II and GIFT-Eval for univariate performance.
- Scientific Case Studies: The model was stress-tested on domain-specific data including ECDC RespiCast (epidemiology), German CPI (macroeconomics), FLUXNET (carbon flux), and CAMELS-DE (hydrology).
Key Insights and Ideas
- Context is King: The model doesn't need to be "taught" the physics of a system. Through group attention, it can identify that a store is closed (regime shift) or that a promotion is driving sales simply by looking at the provided covariates in the context window.
- Synthetic Data Generalizes: A major "aha" moment is that a model trained largely on simulated multivariate dynamics can match or beat specialized models in complex real-world fields like carbon flux prediction or infectious disease tracking.
- Cross-Learning in Univariate Tasks: Interestingly, group attention helps even in univariate benchmarks. By treating multiple related univariate series as a group, the model can "cross-learn" shared patterns, which is particularly effective when historical data for a specific series is scarce.
Key Results
- State-of-the-Art Accuracy: On the
fev-benchbenchmark, Chronos-2 improved skill scores by 5.2 percentage points over the strongest TSFM baselines and increased win rates by 10 percentage points. - Extreme Speedup: Because it requires no task-specific training or tuning, Chronos-2 is 50x faster than the best point forecasters (like CatBoost) and 1000x faster than the best probabilistic forecasters (like TFT) in end-to-end runtime.
- Domain Success: In scientific case studies, it matched or surpassed specialized models. For example, in carbon flux prediction, it achieved the lowest RMSE across multiple energy fluxes without any domain-specific training.
Limitations and Future Work
Despite its strengths, the authors note several frontiers:
- Target Constraints: The model currently only supports real-valued targets (no categorical targets).
- Static Metadata: It handles time-varying covariates well but does not yet natively integrate static metadata (e.g., a basin's soil composition in hydrology).
- Joint Distributions: While it provides marginal quantiles for each variable, it doesn't output a joint predictive distribution, meaning it cannot natively generate sample trajectories that are coherent across both time and dimensions.
- Multimodality: A promising future direction is extending the model to support natural language inputs, allowing it to ingest qualitative data like central bank statements or epidemiological reports.
Summary updated