Forecasting models for systems with hundreds of sensors face one recurring design decision. They can treat every sensor as its own series, which is called channel independence, or they can let sensors exchange information, which is called channel mixing. Channel independence resists noise but cannot see that a traffic jam at one sensor predicts a jam downstream. Channel mixing can see that coupling, but it also has room to memorize chance correlations. The literature does not agree on which choice is right. PatchTST argues independence is the safer default, while CSformer, SOFTS and Adapformer report that mixing wins. No paper in this area says when each one wins. Today that is settled after the fact, by training both and comparing.
We propose to map the crossover jointly as a function of three quantities: training history length $S$, number of variables $C$, and the amount of mixing capacity, measured as the rank $r$ of a cross-channel bottleneck. We also propose to test whether a cheap linear diagnostic, computed in seconds before any deep model is trained, predicts where on that map deep mixing beats deep independence. The contribution is a predictive account of the complexity trade-off for high-dimensional variable scaling, plus a rule for setting mixing capacity that can be checked against data. It is not another mixing architecture.
Core Claim
The benefit of cross-channel mixing comes down to two terms that can be estimated in advance. The first is how much predictive signal the other channels carry beyond a channel's own past. The second is the variance cost of learning a rank-$r$ interaction from $S$ time steps across $C$ channels. Standard parameter counting for reduced-rank regression gives a rough form for the expected gain of rank-$r$ mixing over channel independence:
$$ \Delta(r; S, C) \approx G(r) - \kappa\,\sigma^2\,\frac{r\,(C + d)}{S}, $$
The symbols are defined as follows. $G(r)$ is the reduction in error from the best rank-$r$ cross-channel term. $\sigma^2$ is the residual noise level. $d$ is the per-channel feature width. $\kappa$ is a single constant fitted once per model family.
Both $G(r)$ and $\sigma^2$ can be estimated by fitting two ridge regressions on the training split, one channel-independent and one with a reduced-rank cross-channel term. We claim three things. First, the sign of $\Delta$ predicts whether deep mixing models (iTransformer, SOFTS-style cores) beat deep independent models (PatchTST, DLinear) on held-out datasets and data budgets. Second, the $r$ that maximizes $\Delta$ is a good mixing rank for the deep model. Third, the crossover sample size grows roughly linearly in $C$ at fixed coupling strength. That last claim directly answers the open question in the research file of whether the threshold scales linearly, quadratically or exponentially with variable count.
Relation to Prior Work
The motivation comes from complexity-crossover studies outside forecasting. Neural network language models for low resource languages located a training-size threshold below which recurrent models lose to feed-forward ones. StockMixer, an MLP-based stock forecaster (StockMixer), and studies of recommendation state encoders (State Encoders in RL for Recommendation) show that constrained models win on small, noisy data. All of these locate the crossover empirically and in one dimension, sample size. We add the variable count and a mixing-rank axis, and we try to predict the boundary rather than only observe it.
Forecasting papers provide the two endpoints and the tension we are trying to resolve. Are Transformers Effective for Time Series Forecasting? showed that one-layer channel-independent linear models (DLinear, NLinear) beat Transformers by 20–50%. PatchTST made channel independence the default for Transformers. On the other side, iTransformer attends across whole-series tokens, one per variable. CSformer moves from independence to mixing over training with a curriculum. Adapformer mixes each variable with only its top-$k$ most similar neighbors. All three show mixing helping on their chosen benchmarks at full data. None reports how the comparison changes as $S$ shrinks or $C$ grows, and Adapformer's $k$ and rank are fixed hyperparameters.
The method draws on bottleneck mixers. SOFTS pools all channels into one global core and broadcasts it back, at linear cost in $C$. StockMixer routes stocks through $m$ learned market factors. Think Globally, Act Locally uses global low-rank factors. We reuse this aggregate-redistribute design and generalize it to $r$ cores, so that a single backbone covers the full range from independence ($r=0$) to near-full mixing. That makes "mixing capacity" a single knob we can measure.
One result constrains the design. MTS-UNMixers underperforms on Traffic's 862 channels, which suggests that low-rank or global pooling can underfit dense local spatial structure. Our account predicts this case: $G(r)$ would keep rising at large $r$. We treat it as a check the predictor must pass, not as an exception to explain away.
Research Program
We build one backbone: a PatchTST-style per-channel temporal encoder, followed by a rank-$r$ aggregate-redistribute mixing block. With $r=0$ it is pure channel independence. With $r=1$ it is close to SOFTS. With large $r$ it approaches dense mixing. iTransformer is kept as a separate dense reference point.
We then sweep three factors. Training history is truncated to $S \in$ {5%, 10%, 25%, 50%, 100%} of the standard split. Variable count is varied by random channel subsets $C \in$ {16, 64, 256, full} of PEMS-07 (883 channels), Traffic (862), ECL (321) and Solar (137). Mixing rank takes values $r \in$ {0, 1, 4, 16, 64}. Seeds are repeated throughout.
Alongside the sweep we compute the linear diagnostic $\Delta(r;S,C)$ on each training split. $\kappa$ is fitted on a subset of datasets and frozen before evaluation on the rest.
A synthetic suite controls the ground truth. It contains clustered systems with a known number of coupled groups, known coupling strength and adjustable noise. On this suite we can check that $G(r)$ recovers the true coupling rank. We can also test the multi-cluster failure mode the research file raises for single-core SOFTS.
Baselines
The naive reference points are the last-value Repeat forecast and DLinear. Repeat beat Transformers on Exchange-Rate in Are Transformers Effective for Time Series Forecasting?. DLinear is the cheapest strong channel-independent model, with about 140K parameters on Electricity according to the same paper. Any mixing model that fails to beat these at low $S$ confirms the crossover rather than refuting the proposal.
The channel-independence competitor is PatchTST. It is the strongest argument that mixing is unnecessary, and its paper reports a 21% MSE reduction over prior Transformers.
The strongest mixing competitors are Adapformer and SOFTS. For Adapformer, rank and $k$ are chosen by validation search, and it reports a 32.9% MSE improvement over prior models on PEMS-07. SOFTS reports a 13.9% relative MSE decrease on PEMS07 and a 4.4% decrease on Traffic. FACT is included as a convolutional mixing reference; it reports a 23.9% MSE reduction relative to iTransformer on PEMS03. The file contains no absolute per-horizon MSE values on these benchmarks ("no reported result found on Lacuna" for absolute ECL, Traffic, PEMS or Solar MSE), so every baseline will be rerun from released code. SOFTS code is at the GitHub repository listed in its paper.
Fairness rules apply to every model. All models use the same truncated training split at each $S$, the same lookback and horizons (96 to 720), and the same validation tail. Each gets an equal hyperparameter-search budget of 20 configurations and equal GPU hours. Mixing ranks and $k$ are tuned within that budget.
The key ablation compares three ways of choosing the rank for our backbone. One uses the diagnostic's predicted $r$ with no search. One uses a validation-tuned $r$ under the same budget. One uses a fixed $r$. We also run the diagnostic with its variance term removed, to show that the $C/S$ penalty, not just the signal estimate, carries the prediction.
"Beating" the strongest baseline is defined as follows. Choosing $r$ by the diagnostic must match validation-tuned Adapformer and SOFTS within one seed standard deviation at full data, and beat them on average MSE in the $S \le$ 10% regime. Separately, the diagnostic must predict the sign of the deep mixing-versus-independence gap on held-out dataset–budget cells with at least 80% accuracy.
Why not just…?
Why not just tune the mixing rank or $k$ on a validation set? This is the real threat. With enough data it will probably choose as well as our rule does. Our claim is narrower. When $S$ is small, the validation tail is short and drawn from a shifted period, so validation choices become noisy. Validation search also costs a full deep-training sweep per configuration, while the diagnostic costs a few ridge fits. The rank-selection ablation settles this: we report the gap between validation-chosen and diagnostic-chosen rank as a function of $S$. If validation choice wins everywhere, the rule has no practical value. The crossover map and its scaling with $C$ would still stand as findings.
Why not just use channel independence everywhere, as PatchTST and DLinear suggest? Mixing models report large gains on PEMS, such as SOFTS' 13.9% and Adapformer's 32.9% on PEMS-07, and FACT reports 15–30% gains from adding cross-variable blocks to PatchTST. Independence clearly leaves signal unused on coupled systems. What is missing is the location of the line between the regimes. Our sweep tests exactly whether independence wins at low $S$ and high $C$, and if it does, by how much.
Why not just use iTransformer with random channel subsampling? It scales quadratically in $C$ at inference. Subsampling also breaks pairs of interacting variables during training. More importantly, it does not answer the question: dense attention sits at one end of the rank axis. Including it shows whether dense mixing is ever optimal, which the Traffic case suggests it might be, and whether the diagnostic predicts that case.
Why not just use CSformer's curriculum from independence to mixing? A curriculum hedges against the trade-off during optimization, but it still commits to quadratic channel attention and does not say how much mixing the data supports. We include CSformer at $C \le 321$, where its memory cost is tractable. It counts against the proposal if it dominates in the low-$S$ regime.
Decisive Checks
Three outcomes would falsify the proposal. The first is that the diagnostic's sign accuracy on held-out cells is near chance. The second is that the fitted crossover $S^*$ shows no consistent dependence on $C$ across the four real datasets and the synthetic suite. The third is that on synthetic clustered systems the best deep rank does not track the true number of coupled groups. The proposal survives if all three hold: the measured crossover moves roughly in proportion to $C$, the frozen-$\kappa$ diagnostic generalizes across datasets, and the rule-chosen backbone is competitive with tuned SOFTS and Adapformer at a small fraction of their tuning cost.
References
- Neural network language models for low resource languages
- StockMixer: A Simple Yet Strong MLP-Based Architecture for Stock Price Forecasting
- State Encoders in Reinforcement Learning for Recommendation: A Reproducibility Study
- Are Transformers Effective for Time Series Forecasting?
- A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
- iTransformer: Inverted Transformers Are Effective for Time Series Forecasting
- CSformer: Combining Channel Independence and Mixing for Robust Multivariate Time Series Forecasting
- Adapformer: Adaptive channel management for multivariate time series forecasting
- SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion
- Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting
- MTS-UNMixers: Multivariate Time Series Forecasting via Channel-Time Dual Unmixing