Most forecasting models only look at the past of the series they predict. They therefore miss heatwaves, policy changes, and supply shocks until those events already show up in the numbers. The usual fix is to feed the forecaster news or reports as extra input, called exogenous context. That fix has a credibility problem. An audit of popular text-plus-series forecasters found that they do not read the text at all: replacing it with empty, shuffled, or random strings changed error by less than 0.5%. The reported gains came from a leaked numeric column (Semantics or Structure?). The hard part is not finding a way to inject text. It is getting a model to decide, case by case, whether a piece of context should change the forecast at all, and if so, how.
We propose that a small language model, around 1.5B to 7B parameters, write a short executable "intervention program" from the text. The program is either the explicit choice NONE, which leaves the series unaffected, or a few parametric shocks. Each shock is a spike, step, or bell curve with an onset, duration, sign, and size. The program is turned into a future covariate curve and fed to a frozen pretrained forecaster. The language model is trained with reinforcement learning, and its reward is how much the program lowers probabilistic forecast error on the realized future. This turns forecasting into a checkable task, much like unit tests for code. Choosing NONE always earns zero reward, so a model that injects shocks it cannot back up gets punished. Because the program is discrete and readable, any gain it produces can be checked for whether it depends on what the text says.
Core Claim
Our claim is that excluding a covariate should be something the model learns to do, not a fixed design choice. Three things are needed to make text-conditioned forecasting both content-sensitive and safe against spurious context. The text should reach the forecaster only through a narrow channel the model can be held accountable for. The channel needs an explicit "exclude" option. And the program writer should be rewarded on realized forecast improvement.
Two further claims are secondary and we will test them, not assume them. First, a language model whose reasoning was trained on math transfers better to unseen forecasting domains. The plausible mechanism is that reading a news item and committing to "a step of this size starting here" is a form of variable isolation, a skill that transfers from math to code and science in CPL. Second, splitting the task into two roles helps. One role is a judge that only decides whether the context is relevant. The other writes the program. This split mirrors the critic agents that cut confirmation bias in LEGO.
Relation to Prior Work
The closest existing systems inject text or covariates through continuous channels. DoubleCast fuses text into a T5 decoder with cross-attention and wins 71% of comparisons against unimodal Chronos on the verified CAF-7M test set. UniCA projects text, images, and tables into hidden series. TiMi prompts a frozen language model once and uses the output to steer which expert layers a Transformer uses. All three let the text affect the forecast through learned dense pathways that the audit literature shows can quietly ignore content. None of them trains the language component on forecast outcomes.
On the covariate side, two papers supply our method. ChronosX gives the adapter that takes known future covariates into frozen Chronos, and its synthetic suite of spikes, steps, and bell curves gives our shock vocabulary. TFMAdapter motivates the exclusion option: it already falls back to the univariate forecast when covariates look unreliable, but it does so with a fixed uncertainty filter on numeric covariates, not a learned decision about text.
KITE handles spurious covariates with statistical guardrails such as Pearson correlation and Granger causality. It still needs numeric future covariates to be given. Our programs create those covariates from text.
Towards Time Series Reasoning with LLMs shows that a specialized 7B model can reason about series better than GPT-4o. It reaches 0.698 accuracy on Chinatown against 0.347 for GPT-4o, but it cannot forecast. We keep the generating of numbers inside a frozen forecaster and give the language model only the decision about shocks.
MAFS splits forecasting across agents by horizon and signal property but never looks at exogenous context. Our two-role split moves decomposition to the relevance question. We expect the gain to be small, in line with MAFS's modest 6.35% MSE reduction and its saturation around 16 agents.
Research Program
The forecaster is frozen Chronos with a ChronosX-style output injection block. We pretrain that block once on synthetic series with known injected shocks and then freeze it. After that, the executed program is the only way the text can affect the forecast.
A program $\pi$ is a list of at most $K$ shocks, or NONE. It is rendered into a future covariate curve $c_\pi$. The reward for context $x$, history $y_{\le t}$, and realized future $y_{>t}$ is
$$r = \frac{\mathrm{CRPS}(F_0, y_{>t}) - \mathrm{CRPS}(F_{c_\pi}, y_{>t})}{\mathrm{CRPS}(F_0, y_{>t})} - \lambda |\pi|.$$
Here $F_0$ is the unconditioned Chronos forecast, $F_{c_\pi}$ is the forecast given the program's covariate, CRPS is the continuous ranked probability score (a standard error measure for probabilistic forecasts), $|\pi|$ is the number of shocks, and $\lambda$ is a small per-shock cost. With this reward, NONE earns exactly zero, and an unjustified shock earns a negative reward.
Training happens in two stages. In the warm start, for each training example we run a search over shock parameters to find the program that best explains the forecast residual. We then label the example NONE whenever even the best program improves CRPS by less than a margin. A Qwen2.5 policy is fine-tuned to predict these labels from the text plus a compact rendering of the history. In the second stage, we run Group Relative Policy Optimization (GRPO) with the reward above, following the recipe AutoTriton used for execution-rewarded code generation.
We vary three factors, one at a time. The first is the initialization: plain Qwen2.5 versus Qwen2.5 with a math-derived reasoning vector added, following Reasoning Vectors. That paper uses the same model family, so the weight arithmetic is valid. The second is single-policy versus two-role, where a judge outputs relevant or not relevant before a separate writer produces the program. The third is which domains are held out from training.
Baselines
The naive reference point is frozen Chronos with no context. Every gain is measured against it. We also report how often the proposed system matches Chronos exactly by choosing NONE.
The strongest existing method is DoubleCast. It is the only system in the file trained end to end on CAF-7M, and it shows real content sensitivity: its performance plummeted when we swapped in text from other windows. Its reported result is a 71% win rate over Chronos on the 904-sample verified CAF-7M test set (Overcoming the Modality Gap). We will use its released training data, the same train/test split, and our CRPS evaluation. Beating it means a higher win rate over Chronos on the verified set and lower mean CRPS. It also means being clearly more robust on the distractor-augmented set described below, where we expect dense fusion to degrade.
The second baseline is TiMi-style single-pass distillation from a frozen language model, as described in TiMi. TiMi reports a 29.57% MSE reduction over PatchTST on Time-IMM. We will run it with the same Qwen2.5 backbone to control for model size.
The third baseline is a frontier model prompted zero-shot to write the same program format, executed through the same frozen adapter. We found no reported result on Lacuna for this baseline.
The key ablations isolate our main ingredients. We remove the NONE option and the per-shock cost. We replace the program with a continuous text embedding fed to the same adapter, which is the UniCA-style channel. And we stop after the warm start, skipping reinforcement learning.
Why not just…?
Why not just prompt a frontier LLM to write the program? This is the most serious threat. The program format may do most of the work, and a large model may already be good at turning news into shocks. Two pieces of evidence argue for specialization, but only in other tasks. In AutoTriton, an RL-trained 8B model reached 39.16% execution accuracy on TRITONBENCH-T against 15.06% for GPT-4o. In Towards Time Series Reasoning, GPT-4o read raw series poorly. Neither result is on forecasting. If the zero-shot frontier baseline matches our trained policy, the contribution shrinks to the channel and the NONE option, and we will report it that way.
Why not just train the DoubleCast-style fusion model end to end? It already shows real content use. Its numbers-only GIFT-Eval result, nearly identical to Chronos, is the behavior we also want. The open question is spurious context: text that is real and on-topic but irrelevant to the forecast. Dense cross-attention has no built-in way to ignore that kind of text. The distractor-augmented test measures exactly this. If DoubleCast holds up there, our method's advantage is limited to interpretability.
Why not just supervised fine-tuning on the search-fitted programs, without RL? This baseline is cheap, and it may be enough. The warm-start-only ablation tests it directly. RL has to earn its place by improving CRPS and the calibration of the NONE decision on held-out domains, where the fitted labels may not generalize.
Proposed Experiments and Decisive Checks
The primary benchmark is CAF-7M (ServiceNow/CAF_7M), using leave-domains-out splits across its 11 domains. The verified test set was itself filtered for cases where GPT-5.2 judged the context to be helpful. That filter biases the set toward cases where context matters, so we build a distractor-augmented version. In half the samples, we replace the context with on-topic text from a non-overlapping window of the same series. The correct behavior on those samples is NONE.
Secondary evaluations use three benchmarks. On Time-MMD with prior_history_avg removed, the audit attributed essentially all prior gains to that column, so any gain we show there must come from the text. On Time-IMM, we test asynchronous text. On GIFT-Eval, which has no text, we check that the system does no harm.
We have three decisive checks. The first is the content-sensitivity audit protocol from Semantics or Structure?. Unlike the audited models, our CRPS gain should disappear under empty, shuffled, or cross-domain text, and the NONE rate should rise.
The second is held-out-domain transfer. If math-reasoning initialization matters, it should widen the gap over plain initialization on unseen domains, not on the domains seen in training.
The third is the distractor test. A learned exclusion policy should keep CRPS within noise of Chronos on distractor samples while keeping most of its gain on real ones. Failing this check would undercut the core claim regardless of headline accuracy.
We will also compare the two-role design against the single policy. That comparison stands or falls on the NONE decision's precision and recall, not on average error.
References
- Semantics or Structure? Auditing Text Sensitivity in Multimodal Time-Series Forecasting
- CPL: Critical Plan Step Learning Boosts LLM Generalization in Reasoning Tasks
- LEGO: A Multi-agent Collaborative Framework with Role-playing and Iterative Feedback for Causality Explanation Generation
- Overcoming the Modality Gap in Context-Aided Forecasting
- UniCA: Unified Covariate Adaptation for Time Series Foundation Model
- TiMi: Empower Time Series Transformers with Multimodal Mixture of Experts
- ChronosX: Adapting Pretrained Time Series Models with Exogenous Variables
- TFMAdapter: Lightweight Instance-Level Adaptation of Foundation Models for Forecasting with Covariates
- KITE: Knowledge-Guided Probabilistic Modeling for Time Series Forecasting with Exogenous Variables
- Towards Time-Series Reasoning with LLMs
- Many Minds, One Goal: Time Series Forecasting via Sub-task Specialization and Inter-agent Cooperation
- AUTOTRITON: Automatic Triton Programming with Reinforcement Learning in LLMs
- Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic