Dependency-Alignment Proxies for Cheap Architecture Search in Multivariate Forecasting

When a model forecasts many sensor readings at once, such as hundreds of traffic loops or hundreds of electricity meters, its most consequential design choice is how information flows between channels. It might keep every channel separate, mix all pairs, or mix groups of channels. The right answer changes from dataset to dataset. Searching for it automatically is expensive. The best current search method for correlated time series still spends 53.5 GPU hours on PEMS08 and 184.5 on PEMS03 (AutoCTS+). The cheap "zero-cost" scoring rules that rank vision architectures without training fail on time series forecasting (DARTS-TS).

We propose a cheap score built for this specific decision. After only a few dozen training steps, we measure how strongly each predicted channel's output depends on each input channel. We then check whether that sensitivity pattern lines up with the cross-channel predictability actually present in the data. An architecture that routes its gradient pathways through channel pairs that genuinely predict each other should score high. One that spreads its attention over unrelated channels, which is the "spurious mixing" failure documented by CSformer, should score low. The claim is that this data-aware, channel-aware score ranks candidate architectures better than validation loss measured at the same tiny compute budget. The payoff would be pruning most of a search space before any architecture is trained for real.

Core Claim

Vision proxies fail on forecasting models for a plausible reason: they are blind to the two-axis structure of the data. Scores such as gradient saliency, Synflow, or activation-pattern counts describe a network's capacity in general. They never ask whether that capacity points at the dependencies the data contains. We define a data-derived dependency matrix $D$, where $D_{ij}$ is the fraction of variance in channel $i$'s next-horizon values explained by channel $j$'s past. $D_{ii}$ covers a channel's own past. The off-diagonal terms are estimated by ridge regression on a training slice, beyond what the channel's own past already explains. We also define a model sensitivity matrix $S$, where $S_{ij}=\mathbb{E}_x \lVert \partial \hat{y}_i / \partial x_j \rVert$ is the average magnitude of the gradient of the forecast for channel $i$ with respect to the input history of channel $j$, after $k$ optimizer steps. The score is

$$A_k = \frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{N} \frac{S_{ij}}{\sum_{l} S_{il}}\, D_{ij},$$

the average amount of real predictive dependency that the model's input sensitivity is routed through, where $N$ is the number of channels. A channel-independent model has only diagonal sensitivity, so its score reduces to the average self-predictability. This makes the long-running "channel independence versus channel mixing" debate a comparison on one axis instead of a special case. A mixing model beats a channel-independent one only if it concentrates its off-diagonal sensitivity on pairs with large $D_{ij}$.

We are explicit about one weakness. At random initialization, learned-graph and attention modules spread sensitivity almost uniformly. The score at $k=0$ therefore measures only how broadly an architecture can mix, not what it will learn. For this reason the main object is $A_k$ at small $k$, on the order of 50 to 200 minibatch steps. $A_0$ is kept as a genuinely training-free variant whose success or failure is itself informative.

Relation to Prior Work

Four papers motivate the proposal by exposing the gap. DARTS-TS searches a hierarchy of sequence-style and flattened cells with a differentiable supernet. It reports that vision zero-cost proxies fail on forecasting, but it does not construct a replacement; we do. AutoCTS+ replaces the supernet with a pairwise comparator network trained on 5-epoch noisy labels plus a handful of fully trained models, which keeps GPU memory constant. Its search space stays within graph-convolution and transformer blocks, and it still trains hundreds of proxy models. Our score targets that labeling cost directly and extends the space to modern channel-mixing operators. EZNAS evolves proxies from layer statistics on vision benchmarks, reaching a Kendall rank correlation of 0.65 on NASBench-201 CIFAR-10. It also reports that such proxies struggle to separate the top 10% of architectures. That negative result constrains our design: we expect the score to prune, not to pick the winner. Is isotropy a good proxy for generalization shows that embedding isotropy of trained transformers tracks forecasting error. It was used only after training, and we test it as a competing cheap score.

Several papers supply the search space rather than the method. Crossformer attends over time within each channel and then across channels. CSformer shares parameters between channel attention and sequence attention. HyperMixer mixes groups of channels through learned hyperedges and reports a 3.46% average gain when added to DLinear and PatchTST. Prime Attention modulates each channel pair's keys and values with a primer initialized from FFT lead-lag and Pearson statistics. Prime Attention is the closest methodological cousin: it injects data-derived pairwise statistics into the model. We inject similar statistics into the ranking of models.

Research Program

We first build a small ground-truth benchmark, because no tabular benchmark of trained multivariate forecasting architectures exists in the literature we found. The search space crosses temporal backbones with channel operators. The temporal backbones are a linear map, a patch transformer, a temporal convolutional network, and an MLP-mixer. The channel operators are none (channel-independent), a Crossformer-style router, CSformer-style shared attention, Prime Attention, a HyperMixer-style hypergraph mixer, and an adaptive-graph convolution. Width, depth, and lookback length vary within each combination. We sample roughly 500 architectures and train each to convergence on seven datasets spanning weak and strong cross-channel coupling: ETTh1, Weather, Exchange-Rate, Electricity, Solar-Energy, PEMS08, and PEMS03.

On this benchmark we compute $A_k$ for several values of $k$, along with every competing cheap score at matched wall-clock cost. We then run an end-to-end search. The score prunes the space to its top slice, and a SEARCH-style comparator, fine-tuned with fewer early-validation labels, chooses within that slice. The analysis side asks when the score works. We predict its advantage over plain short-run validation loss grows with the off-diagonal mass of $D$. On nearly independent data like Exchange-Rate, all methods should agree that channel independence wins. On traffic data they should diverge.

Baselines

The naive reference point has two parts. The first is to do no search and train a channel-independent PatchTST and DLinear on each dataset. The second is random search with the same total GPU budget as our method. If our pipeline cannot beat random search at equal compute, the score has no value. Parameter count used as a ranking score also belongs here, since capacity alone often correlates with accuracy in proxy studies. It costs nothing and must be beaten on rank correlation.

The strongest existing method is SEARCH from AutoCTS+, which reports 53.5 GPU hours on PEMS08 and 184.5 on PEMS03. We run it on our search space with its published protocol of 5-epoch noisy labels and a few clean models, on identical data splits and hardware. We report GPU hours with the same accounting as ours. Beating it means reaching equal or lower test MAE on PEMS03 and PEMS08 with at most half its search GPU hours, or clearly lower MAE at equal hours.

For absolute accuracy on traffic, the searched-model reference is SNAS4MTF at 3.43 MAE on METR-LA 60-minute forecasting. We include METR-LA in the final evaluation for this comparison only. The file gives no reported rank correlation for vision zero-cost proxies on forecasting tasks, only the qualitative failure in DARTS-TS, so no reported result was found on Lacuna for that. We therefore re-run Synflow, NASWOT, and SNIP ourselves on the benchmark. DARTS-TS is run as a supernet baseline where memory allows, and any out-of-memory failures on PEMS03 are reported rather than hidden.

The key ablations isolate the data term. The first replaces $D$ with a uniform matrix, which leaves a pure "how much does it mix" score. The second replaces $D$ with a shuffled-channel version. The third is validation loss after the same $k$ steps. If $A_k$ does not beat all three, the dependency alignment is not what carries the signal.

Why not just…?

Why not just use SEARCH's 5-epoch early validation, which already works? This is the most serious threat. We suspect short-run validation has a specific bias: channel-independent models converge fast, while mixing models often look worse early and overtake later, so short runs may systematically under-rank mixers. That is a hypothesis, not established evidence. The matched-compute comparison of $A_k$ against validation loss at step $k$ and at 5 epochs settles it, stratified by channel operator. If early validation ranks as well per GPU hour, we will report the proxy as unnecessary and keep only the benchmark.

Why not just choose channel independence or mixing from data statistics, with no search? A rule such as "mix if the off-diagonal mass of $D$ exceeds a threshold" uses the same data term and none of the architecture term. It is a real competitor. We run it as a baseline that selects the best-performing operator family by that rule, with a threshold tuned on held-out datasets. The proposal survives only if choices within the mixing families, such as hypergraph versus Prime Attention versus graph convolution, matter enough that the model-side sensitivity term adds accuracy beyond the rule.

Why not just run PatchTST or a foundation model and skip search altogether? The evidence cuts against this. HyperMixer reports that adding channel mixing improves channel-independent backbones by 3.46% on average. TSFM-Bench finds that no foundation model dominates and that most struggle on strongly coupled data. These models remain the do-nothing baseline, and the evaluation includes datasets where they should win.

Why not just evolve a proxy EZNAS-style on time series statistics? Evolution over generic layer statistics could rediscover something like our score, but it needs a ground-truth benchmark to evolve against. Our benchmark provides one, and we include an evolved proxy trained on four datasets and tested on the other three. That split checks whether our hand-derived score transfers better than a fitted one.

Proposed Experiments and Decisive Checks

The first check is ranking quality on the benchmark. We measure Kendall rank correlation and top-10% recall of each cheap score against fully trained test MSE and MAE per dataset, at matched GPU seconds per architecture. The prediction is that $A_k$ beats validation loss at step $k$ on the four strongly coupled datasets (Electricity, Solar-Energy, PEMS08, PEMS03), and ties it on ETTh1 and Exchange-Rate.

The second check is the ablation trio of uniform $D$, shuffled $D$, and $k=0$. Shuffled $D$ should collapse the gain. If it does not, the data term is doing nothing.

The third check is the end-to-end search. We report final test MAE, MSE, and search GPU hours on PEMS03 and PEMS08, compared with SEARCH's reported 184.5 and 53.5 hours and with our re-run of SEARCH, and on METR-LA compared with SNAS4MTF's 3.43. We also report which channel operator each dataset's winning architecture uses. This tests the analysis claim that the search picks channel independence where $D$ is nearly diagonal and higher-order mixing where group structure exists.

A negative outcome would still be informative. If $A_k$ fails but the benchmark shows mixers are systematically under-ranked by early validation, that finding alone changes how comparator labels should be collected.