Spectral Error Covariance for Calibrated First-Passage Survival Curves

Many "will an event happen in the next H steps?" questions in time series are really threshold questions about a forecast. Will PM2.5 exceed a limit within six hours? Will temperature cross a heat-spike level? Will volatility jump above a regime cutoff? If a model gives a full joint distribution over the next H values, the chance that the series has crossed the threshold by step k is a survival curve. Survival analysis is the study of time-until-event distributions, and here the curve is non-increasing in k for free, because the set of paths that have already crossed can only grow. The catch is that this curve depends almost entirely on how forecast errors at different future steps are correlated. Standard training with mean squared error (MSE), or with independent per-step Gaussians, treats those errors as independent. That gives the model many independent chances to cross, which inflates event probabilities. Learning one fixed correlation matrix for the whole dataset helps, but it cannot tell a calm window from a turbulent one.

We propose to predict, for each input window, the full error covariance across future horizons. The covariance is parameterized in the frequency domain: the error process is represented by a per-window error spectrum, meaning how much forecast-error power sits at slow versus fast oscillations across the horizon. A frequency-domain attention module reads the input window and outputs that spectrum. This makes the covariance input-dependent and cheap to invert. The resulting joint forecast is turned into monotone first-passage survival curves by sampling, with no constrained network and no event labels at training time. The claim to test is that these curves are better calibrated than both static-covariance forecasts and directly trained survival heads, especially when event labels are scarce or the threshold changes at test time.

Core Claim

Covariance across horizons is the missing ingredient that makes forecast-derived event probabilities trustworthy, and that covariance must depend on the input. A frequency-domain parameterization makes input-dependent covariance cheap enough to train with an exact Gaussian likelihood at realistic horizons (H = 96 and beyond). It also gives a natural role to spectral attention: mapping the spectral content of the input window to the spectral content of the coming error.

Three lines of prior work motivate this.

Direction 1 documents that errors are state-dependent:

  • Continuous-time variational filters with fixed uncertainties fail when fluid systems move between calm and turbulent regimes (Blömker et al.).
  • Photovoltaic forecast errors change distribution across the day (Cheng et al.; Touron et al.).
  • Solar radiation forecasts degrade on overcast days (Wang et al.).

The closest method is Quadratic Direct Forecast (QDF). QDF replaces MSE with a loss weighted by a learned inverse covariance across the 96 horizons, trained by bilevel optimization. Its learned matrix is static per dataset, and the authors name input-conditioned covariance as open. QDF also uses the matrix as a loss weighting to improve point forecasts, not as a predictive distribution. We differ on both counts: our covariance changes with every input window, and it is the actual predicted uncertainty, scored by likelihood.

The dynamic mixture model for traffic lets mixture weights vary over time but keeps each component's covariance fixed. We let the covariance itself move.

The Model

A backbone forecaster produces a mean path $\mu(X) \in \mathbb{R}^H$ from input window $X$. We model the horizon error as Gaussian with covariance

$$\Sigma(X) = D(X)\, C^\top \mathrm{diag}\big(p(X)\big)\, C\, D(X),$$

The symbols are:

  • $C$ is the orthonormal type-II discrete cosine transform (DCT) over the $H$ horizons.
  • $p(X) \in \mathbb{R}^H_{>0}$ is the predicted error spectrum, normalized so its mean is one.
  • $D(X)$ is a diagonal matrix of per-horizon error scales.

The DCT part encodes a stationary correlation shape. This relies on a classical fact: covariances of stationary processes are approximately diagonalized by Fourier-type bases, and the DCT handles window edges better than the discrete Fourier transform. $D(X)$ supplies the non-stationary growth of error with lead time that QDF found to matter.

Inversion and log-determinant cost $O(H \log H)$. Sampling is equally cheap: draw $z \sim \mathcal{N}(0, \mathrm{diag}(p))$, apply $C^\top$, scale by $D$, and add $\mu$.

The spectrum $p(X)$ comes from a local spectral attention module in the style of SAAM. Moreno-Pino et al. used SAAM to filter hidden states of autoregressive forecasters. We reuse that mechanism for a different job: emitting the error spectrum rather than a filtered state. We do not hardcode a low/high band split of the kind TF-FACE uses; every coefficient gets its own learned power. Training is the exact joint Gaussian negative log-likelihood, a proper scoring rule for the joint forecast.

For a threshold $c$ applied to the target channel, the survival curve is

$$S(k \mid X) = P\big(\max_{j \le k} Y_{t+j} < c \mid X\big).$$

We estimate it by the running maximum over a few thousand samples. Each sample path's running maximum is non-decreasing, so the estimated curve is monotone exactly, not just in expectation. The discrete event density is $S(k-1)-S(k)$.

This is where the survival direction enters. Monotonicity is the property the literature works hard to impose through:

In our construction it comes from the geometry of first passage instead. The modeling burden moves from the survival head onto the covariance.

Research Program

The work has three parts.

Synthetic study. We generate autoregressive series whose error autocorrelation and volatility switch between regimes, so the true first-passage curve is computable by brute-force simulation. This lets us measure how far independent, static, and dynamic covariances each move the curve from the truth. We expect independent errors to overstate crossing probability at long horizons. A static matrix should be miscalibrated in opposite directions in the two regimes.

Integration. We attach the spectral covariance head to the backbones QDF used, specifically PatchTST, iTransformer, and Fredformer, on ETT, ECL, and Weather at H = 96. We report joint likelihood and point accuracy against QDF.

Event study. We apply the same trained forecasters, without retraining, to threshold-crossing events on the HEPA spike benchmarks (HEPA): Weather heat spikes, Beijing-AQ PM2.5 spikes, and VIX spikes. Where HEPA's event definitions turn out not to be threshold crossings of a forecastable channel, we define threshold events ourselves and report both. Windows whose events fall beyond the horizon are treated as censored, so evaluation uses the right-censored log-likelihood. Rindt et al. prove this is strictly proper, whereas time-dependent concordance and integrated Brier scores are not. We also report calibration in probability space. That matches HEPA's own stated weakness: good ranking but poor absolute calibration. Following Lillelund et al., C-index appears only as a secondary diagnostic.

Baselines

Naive references. The first is the backbone trained with MSE, with an event declared when the point forecast crosses the threshold. This produces a step-function curve and shows what ignoring uncertainty costs. The second is the backbone with an independent Gaussian per horizon, trained by Gaussian likelihood, which isolates the cost of ignoring cross-horizon correlation.

Static-covariance competitor: QDF. QDF is run with the authors' bilevel procedure on the same backbones, data splits, and hyperparameter search budget. Its learned matrix is then used as a fixed predictive covariance, with the scale refit on validation data so it is a fair density, and the same sampling estimator gives event curves. QDF reports best results in 39 of 40 settings on TQNet and MSE reductions of 7.4% for Fredformer and 5.9% for TQNet on ECL. We count as beating it any of three outcomes: matching its point MSE within noise while improving joint likelihood, improving event log-likelihood, or improving calibration on most of the event benchmarks.

Mixture competitor. The dynamic mixture model has dynamic weights over static full-covariance components. We use a single-node version so the Kronecker factorization reduces to a temporal covariance, with K tuned on validation data. This tests whether choosing among a few fixed covariances already captures the state dependence. No reported result was found on Lacuna for its metric values on our datasets.

Strongest direct survival competitor: HEPA's monotone CDF head. It is fine-tuned on event labels at the full label budget and at 2% of labels, the regime where HEPA reports retaining 92% of full-label performance on C-MAPSS. No reported result was found on Lacuna for HEPA's calibration or right-censored log-likelihood on Weather, Beijing-AQ, or VIX. We therefore rerun it ourselves with the same encoder size and tuning budget. As a second direct head, we train a SuMo-net-style positive-weight monotone decoder on the same encoder, which tests whether architectural monotonicity plus a proper loss is enough.

Ablations. One isolates the spectral parameterization: a time-domain hypernetwork that outputs a full input-conditioned Cholesky factor of the 96×96 covariance, with matched parameters. Another isolates the input dependence: at test time each window is given another window's predicted covariance. If shuffling does not hurt, the model is not using the dynamics. A third replaces spectral attention with a plain MLP on the DCT coefficients, testing whether the attention mechanism adds anything.

Why not just…?

Why not just train a monotone survival head directly on the event labels? This is the real threat. On a single fixed threshold with plentiful labels, a direct head trained on the right-censored likelihood may win, because it optimizes exactly the target. Our case rests on two regimes where it should not:

  • Scarce labels. Our model learns from every time step of the regression signal rather than from rare events.
  • Changed thresholds. Our curves are computed for any threshold with no retraining, whereas a direct head must be refit.

The label-budget sweep and the held-out-threshold evaluation settle this. If HEPA's head matches us at 2% labels on unseen thresholds, the forecast route is not worth it, and we would say so.

Why not just use QDF's static covariance? The synthetic regime-switching study predicts a specific failure: a single matrix averages over regimes and miscalibrates each in a different direction. The test-time shuffling ablation and the direct QDF comparison measure whether real data has enough state dependence for this to matter. The evidence of state-dependent errors in photovoltaic, radiation, and weather forecasting suggests it does, but that evidence concerns error variance, not correlation shape. That is an open empirical question, not a given.

Why not just output a full Cholesky factor from a hypernetwork, without any frequency domain? At H = 96 this is feasible, so the spectral parameterization must earn its place through better generalization or longer horizons, not through possibility alone. The matched Cholesky ablation decides it. If the two tie at 96, we extend to H = 336 and 720, where a full factor's output size grows quadratically and the spectral form's grows linearly. If they still tie, the honest conclusion is that dynamic covariance matters and the spectral form is only a convenience. The stationary-correlation assumption inside the DCT is a known limitation, echoing SAAM's own caveat about Fourier stationarity. The Cholesky ablation is also where that limitation would show up.

Decisive Checks

On synthetic data, the proposal succeeds if the dynamic spectral model's first-passage curves track the simulated truth in both regimes while static and independent models diverge in predictable directions.

On real benchmarks, three conditions must hold together:

  • Joint likelihood improves over QDF without losing point MSE.
  • The event right-censored log-likelihood and calibration improve over QDF-derived curves.
  • The shuffling ablation degrades performance, showing that input-conditioning is doing the work.

The method's practical value is established if it matches or beats HEPA's direct head at low label budgets and on held-out thresholds. If it fails all of these against the full-Cholesky hypernetwork, the frequency-domain component is dropped. If it fails against QDF, the dynamic premise itself is wrong.