Per-Channel Reservoirs for Localizing Drift in Multivariate Streams

When a sensor network, a power market or a monitoring system changes behaviour, the change usually touches only a few variables. One sensor degrades, or the relationship between two counties' pollution levels appears. Most drift detectors still raise one global alarm and retrain everything. Methods that do ask where the drift is either treat data as exchangeable tabular batches, which makes them blind to changes in temporal dynamics, or need expensive neighbour searches and conditional density estimates. We propose a cheap streaming alternative built from reservoir computing. A reservoir is a large, fixed, randomly wired recurrent network whose only trained part is a linear output layer. We give each input channel its own small reservoir, so that channel's internal state depends only on its own history. Each channel's next value is then predicted from all channels' states with a closed-form linear readout.

Two kinds of shift can then be read off per channel. If a channel's own dynamics change, only that channel's reservoir states move. If the way a channel depends on the others changes, only that channel's prediction errors move. The first signal localizes input drift. The second localizes mechanism drift. Fixing either one needs nothing more than refitting one linear readout, while every stable channel keeps its history. Because the reservoir states encode whole input histories, both tests are sensitive to changes in autocorrelation and coupling that leave per-feature marginal statistics untouched. That is exactly the regime where current feature-shift localizers fail.

Core Claim

A block-structured reservoir can do three things at a streaming cost linear in the number of channels. It can localize which channels drifted. It can say whether each drift is a change in that channel's own dynamics or in its dependence on other channels. And it can repair the model by refitting only the affected readouts. Our hypothesis is that it beats both marginal-statistic localizers and linear autoregressive residual monitors on shifts that preserve marginals or are nonlinear, without losing on ordinary mean or variance shifts.

Concretely, channel $j$ drives its own reservoir module with state $h^{(j)}_t \in \mathbb{R}^m$, updated as

$$h^{(j)}_t = (1-a)\,h^{(j)}_{t-1} + a\,\tanh\!\big(W_j h^{(j)}_{t-1} + u_j x_{j,t}\big),$$

where $x_{j,t}$ is channel $j$'s value at time $t$, $W_j$ is a fixed random recurrent matrix with spectral radius below one (this gives the fading memory that defines an echo state network), $u_j$ is a fixed input vector, and $a$ is a leak rate. A ridge readout $\beta_j$ predicts $\hat x_{j,t+1} = \beta_j^\top H_t$ from the concatenated states $H_t = [h^{(1)}_t;\dots;h^{(J)}_t]$ of all $J$ modules. Two windowed statistics are monitored per channel. The first is a state-shift score: a linear-kernel two-sample statistic between reference and recent windows of $h^{(j)}_t$. Because the reservoir implicitly defines a kernel on input histories, this amounts to a kernel test on channel $j$'s trajectories. The second is a residual-shift score on $x_{j,t+1}-\hat x_{j,t+1}$. The four combinations (neither, state only, residual only, both) give a per-channel diagnosis.

Why This Is New and Where It Sits

The research file found no paper joining localized drift and reservoir computing. The nearest works each have one half of the idea. ResCP uses an untrained echo state network as a temporal embedder, matching the current state to past states by cosine similarity to reweight conformal residuals. It supplies our core premise that untrained reservoir states are cheap, informative descriptors of non-stationary history, finishing in 52 seconds where a Transformer baseline took over 4,500. However, it compares the whole state vector globally and neither detects nor localizes drift. Locally Connected Echo State Networks impose locality on the reservoir's wiring, a torus grid of neighbourhoods, for speed and memory. That locality is internal and unrelated to input channels, and all of its readout adaptation variants update every weight at once. RCDyM extracts stability measures from a trained reservoir to anticipate global tipping points. It treats the system as one monolith and predicts when collapse happens, not which variable is responsible. The gas-sensor context network of Warner et al. uses a trained recurrent context pathway over labelled batch histories. It does not say which of the 16 sensors is drifting.

On the drift side, the motivating evidence for localization comes from LDD-DSDA, DiwE and ICD3. These show that global detectors miss drift confined to small regions and that global resets throw away valid knowledge. All three rely on $k$-nearest-neighbour or cluster geometry over exchangeable samples, which scales poorly with dimension and ignores temporal order. DataFix and FSL-Net localize shifted features directly. FSL-Net constrains our design in two ways. It reports failure on subtle manipulations that preserve low-order marginals. It also reports that adding attention to model cross-feature correlation cost large amounts of memory with no accuracy gain. We take cross-channel structure into the linear readout rather than the descriptor. Causal-RuLSIF supplies the conceptual move we borrow: test each variable's conditional distribution given its causal parents, so that a mechanism change is separated from a change in the parents. It degrades when parent sets are large because conditional density estimation becomes intractable. Our residual test replaces that density estimate with a linear readout on a fixed nonlinear embedding of all parents' histories.

The file also flags a tension we must face. A reservoir's fading memory carries a disturbance forward in time, which could create "phantom drift" in channels that never changed. Separate modules confine that echo to the drifted channel's own state. Whether the readouts of other channels stay stable when a parent channel's states move is an empirical question. It is the central risk of the proposal, discussed below.

Research Program

The first component is the detector itself. We build the modular reservoir on GPU with modules of size $m$ between 50 and 200. Hyperparameters (spectral radius, leak, input scale) are shared across modules and tuned once on a reference segment by a small random search. This avoids the CMA-ES runs of up to 20 hours per dataset that the LCESN paper reports. Readouts are fitted by ridge regression on the reference period and updated by recursive least squares. For large $J$, each readout is restricted to a sparse set of modules chosen by group lasso on the reference segment. This plays the role of a parent set and keeps the per-step cost at $O(Jm\,s)$ for $s$ selected modules. Thresholds come from permutation or block-bootstrap calibration on the reference segment, targeting a fixed per-channel false-alarm rate.

The second component is selective adaptation. When channel $j$ is flagged for mechanism drift, only $\beta_j$ is refitted on recent data. When a channel is flagged for input drift alone, we test whether downstream readouts that use its module stay accurate. Where they do not, we refit those readouts only. This is the reservoir analogue of LDD-DSDA's "adapt only drifted regions" policy, with no resampling of raw data.

The third component is analysis of the phantom-drift question. We measure how often a pure input shift in channel $i$ raises residual alarms on a channel $j$ whose mechanism is unchanged. We also measure how this rate depends on the size of the shift relative to the reference range, since a linear readout on saturating units must extrapolate. If the rate is high, we test one mitigation. On an input-only alarm, the readouts of downstream channels are temporarily conditioned on a recentred version of the drifted module's state, and we check whether this restores calibration.

Baselines

The naive reference point is a global detector: a single echo state network with one stacked readout predicting all channels, with ADWIN on the summed residual. It sets every channel to "drifted" on alarm and refits all readouts. It tells us what localization buys at all, in both detection and post-drift accuracy. It is run on the same reservoir budget (total neurons $Jm$) with the same calibration.

The strongest existing localizer is FSL-Net, a learned network that summarizes reference and query batches with moments, histograms and empirical distribution functions and outputs per-feature shift probabilities in one forward pass (FSL-Net). It is the strongest competitor because it already does per-feature localization quickly and reportedly with the highest average localization F1 among existing methods. No numeric localization F1 for FSL-Net is reported in the file, so the comparison will be run fresh using the released model. To be fair to a method that sees exchangeable samples, we feed it windows augmented with lagged copies of each channel (the same lag horizon as our reservoir's effective memory) and aggregate lag-feature scores back to channels. DataFix (Adversarial Learning for Feature Shift Detection and Correction) is run with the same lag augmentation on the lower-dimensional benchmarks where its runtime is tolerable. Beating these means higher channel-level localization F1 at matched false-alarm rate on marginal-preserving shifts, with no worse than a small loss on plain mean or variance shifts, at lower wall-clock cost per window.

The strongest linear competitor is a vector autoregression (VAR) with per-channel residual CUSUM, plus per-channel univariate autoregressive models with a windowed test on their coefficients for input drift. It runs on the same windows and uses the same calibration, with lag order chosen by information criterion on the reference segment. It is the honest threat, discussed below. Beating it means a clear margin on nonlinear or coupling-changing drifts and parity on linear ones.

For mechanism localization on the PM10 data we compare against Causal-RuLSIF (Causal Discovery-Driven Change Point Detection in Time Series). No numeric divergence or detection result is reported for it in the file, so the comparison is again run fresh, using its qualitative finding of the Monterey-to-Mono link appearing in February as one check.

Two ablations isolate the key ingredient. The first keeps everything identical but replaces the block-diagonal reservoir with one densely coupled reservoir of the same size, attributing drift to channels by gradient sensitivity of the state test. This tests whether modularity, rather than reservoirs per se, does the work. The second replaces each module with a random-feature map of a fixed lag window (no recurrence). This tests whether fading recurrent memory matters beyond a lag embedding.

Why not just…?

Why not just fit a VAR and watch per-channel residuals? This is the linear version of our method and a real threat. If drift in practice is mostly linear-Gaussian, a VAR will match us at lower cost. The evidence that nonlinearity matters comes from forecasting rather than drift. The LCESN paper found linear LMS filtering far worse than large nonlinear reservoirs on NARMA10. That does not prove the same for detection. Our synthetic suite therefore includes linear drifts where we expect parity and nonlinear drifts (threshold couplings, changes in a nonlinear autoregressive map) where we predict a gap. If no gap appears on real data, we will report that the reservoir adds nothing there.

Why not just run FSL-Net or DataFix on lag-augmented features? Lag augmentation partly cures their temporal blindness, which is why it is our fair baseline. However, FSL-Net summarizes low-order per-feature statistics and its authors report failures on shifts that preserve marginals. Its attempt to model cross-feature correlation with attention ran out of memory without gains. A change in coupling between two channels leaves every lagged marginal intact, so we expect a gap on coupling drift specifically. The coupling-drift experiment decides this. DataFix's iterative retraining, reported at up to 13 hours on high-dimensional data, rules it out for high-rate streams.

Why not just a single global reservoir with per-channel residual monitoring? This is the dense-reservoir ablation. A global reservoir can still give per-channel residuals. However, every channel's state mixes all inputs, so an input shift in one channel moves the whole state. Input drift then cannot be attributed by state, and phantom residual alarms should be more frequent. If the dense version matches the modular one on attribution and phantom rate, modularity is unnecessary and we will say so.

Why not just per-channel ADWIN or KS tests on raw values? These catch mean and distribution shifts cheaply, and we include them. By construction they miss phase-randomized and coupling shifts, which preserve every channel's marginal. They also cannot separate a mechanism change from a parent change.

Proposed Experiments

The decisive experiment is controlled drift injection into real multivariate series from the LCESN forecasting suite: ETTm1, Weather, Electricity and Traffic. Ground truth is known by construction. In each stream we inject drift into a random subset of 5 to 20 percent of channels, using one of four types. The first is a mean or variance shift. The second is a marginal-preserving change in dynamics, made by surrogate resampling that keeps each channel's value distribution but changes its autocorrelation. The third is a coupling change, where one channel's dependence on another is added, removed or made nonlinear. The fourth is a pure parent shift, where an upstream channel's level moves but downstream mechanisms stay fixed. That last type directly measures phantom drift. Metrics are channel-level localization F1 at a calibrated 1 percent false-alarm rate, detection delay in steps, accuracy of the input-versus-mechanism diagnosis, the phantom alarm rate on unchanged downstream channels, post-drift one-step forecasting error under selective versus global refit, and wall-clock time per step. The core claim fails if, on the coupling and marginal-preserving types, the modular reservoir does not beat both lag-augmented FSL-Net and the VAR monitor in localization F1. It also fails if the phantom alarm rate on pure parent shifts exceeds the alarm rate of the mechanism test on genuinely changed channels.

The second experiment is the California PM10 network used by Causal-RuLSIF. We check whether the residual test flags Mono's mechanism around February, consistent with the reported emergence of a Monterey-to-Mono link, without flagging Monterey's own mechanism. We also compare runtime and behaviour as parent sets grow.

The third experiment is downstream adaptation on the gas sensor drift benchmark used by Warner et al., treating the 16 sensors (8 features each) as 16 channels over the time-ordered samples. There is no per-sensor ground truth, so this tests whether selective adaptation helps classification. A classifier readout on the modular states is updated only through the sensor modules our detector flags. It is compared with refitting the whole readout, under the same protocol of five gases with toluene excluded and training on earlier batches before testing on the next. The bar is the reported mean accuracy of 88.2 percent for the feedforward-plus-context model, against 86.9 percent for the skill-only network and 76.4 percent for the SVM ensemble (Warner et al.). That model uses labelled history and end-to-end training. Matching it with an unsupervised localizer and closed-form readouts would be a meaningful result. We will also report which sensors are flagged in each batch, as an interpretable output the context model cannot give.

As a sanity check on low-dimensional streams, we run the method on Elec as an ordered stream with delayed labels. The reference is LDD-DSDA's reported F1 of 0.8747 (Regional Concept Drift Detection and Density Synchronized Drift Adaptation), and DiwE and DriftSurf are run under the same prequential protocol. We expect little advantage there because Elec has few channels. The point is to show the method does not lose on a standard drift benchmark while being far cheaper than $k$-nearest-neighbour region methods in higher dimensions.