An energy-based model gives every input a single number, its "energy," and treats low-energy inputs as likely. Training it by maximum likelihood takes a gradient that is the difference between two averages. The first average is taken over real data. The second, the "negative phase," is taken over samples drawn from the model itself. This note starts from one observation. In a data stream, that same gradient difference is also a classical statistical test for whether the data still fit the model. When the stream is stationary and the model has converged, the expected likelihood gradient on fresh data is zero. When the data distribution drifts, it is not. The obstacle has always been the negative phase: it needs model samples, and Markov chain samplers are too slow to run at stream speed. An amortized sampler removes that obstacle. Such a sampler is a small generator network trained alongside the energy model to produce its negative samples in a single forward pass.
We propose to run a deep energy model and an amortized sampler together on an unlabeled stream. The two compete in a minimax game: the energy network pushes real data down and generated samples up, and the generator chases low energy. We then use the resulting gradient vector twice. Normalized by its own covariance, which is estimated from generator samples, it becomes a multivariate sequential drift statistic. Unnormalized, it is the parameter update that adapts the model. Detection and adaptation come from one computation, with no per-feature tests and no sliding-window encoders. Because the statistic lives in the model's parameter space rather than in raw feature coordinates, it can register changes in correlation structure that leave every individual feature unchanged.
Core Claim
Let $E_\theta(x)$ be the energy network with parameters $\theta$. Let $q_\phi$ be a generator with parameters $\phi$, trained with the entropy-regularized objective of VERA. Let $\psi(x) = \nabla_{\theta_S} E_\theta(x)$ denote the energy gradient restricted to a fixed low-dimensional parameter subset $S$, for example the final layer or a random projection of all parameters. For a window $B_t$ of incoming points at time $t$ we compute
$$ g_t = \frac{1}{|B_t|}\sum_{x \in B_t} \psi(x) \;-\; \mathbb{E}_{x \sim q_\phi}[\psi(x)], \qquad T_t = |B_t|\, g_t^\top \hat\Sigma^{-1} g_t , $$
where $\hat\Sigma$ is the covariance of $\psi$ under generator samples. This covariance is an estimate of the model's Fisher information on $S$. The quantity $g_t$ is, up to sign, the maximum-likelihood gradient on $B_t$. It is also the score of a Rao test of the null hypothesis "the window comes from the current model." $T_t$ is its Hotelling form, which is approximately chi-squared with $|S|$ degrees of freedom under the null if $q_\phi$ matches the model; that standard asymptotic result is where the threshold comes from.
We accumulate $T_t$ with a cumulative-sum (CUSUM) change detector. Below threshold, the model takes a small step along $g_t$, so it tracks slow, incremental drift. Above threshold, it takes larger steps and more generator updates until the statistic returns to its null range. This spends compute only when the stream has moved.
What would be new is the specific shape of this loop: a deep energy model whose drift test is its own likelihood score, made cheap by an amortized sampler and kept honest by calibration. Three neighbors are close but different. RRBM-DD (Adversarial concept drift detection under poisoning attacks) monitors drift with an energy model, but uses a shallow restricted Boltzmann machine trained by Gibbs sampling and detects drift by heuristic trend and Granger tests on reconstruction error, without false-alarm guarantees. AMRC (Minimax Classification under Concept Drift) also solves a minimax problem at each step, but its game is over classification risk with fixed random-Fourier features, and its cost grows as $O(mk^3)$ in the feature count $m$ and the order $k$ of its motion model. It models no input density and uses no sampler. Energy-Based Models for Continual Learning shows that energy training resists forgetting in task-free streams, but its negatives are only labels drawn from the current batch, and it detects nothing.
Research Program
Training on the stream. The first component is the online minimax trainer. The method comes from No MCMC for me. That paper trains the energy model against an entropy-regularized generator instead of Langevin chains, estimates the generator's entropy gradient by self-normalized importance sampling, and reports 2.8x faster training on CIFAR-10 with no divergences. Maximum Entropy Generators supplies the motivation for the entropy term: without it the generator collapses onto a few low-energy regions. We reuse both. Our only change is that the generator is warm-started across stream steps, so it acts as a persistent memory of the current model distribution.
Two known weaknesses we must handle.
The first is the mismatch between the generator and the true Boltzmann distribution. The research file lists this as an open problem, and it biases the null distribution of $T_t$. We do not assume it away. During an initial stationary prefix of real data, we record the empirical distribution of $T_t$ and set the CUSUM threshold from it, not from the chi-squared table. We then measure how much the realized false-alarm rate deviates from nominal as the generator ages.
The second weakness constrains the design. VERA's entropy estimator breaks down when the generator's latent dimension approaches the data dimension, and VERA did worse than Markov chain sampling under some shifts, such as the CelebA out-of-distribution benchmarks. For image streams we therefore run the energy model on features from a frozen encoder, which keeps the generator's dimension low. We report results with and without this choice.
Labels. Labels in streams arrive late or sparsely, so we add a joint energy over inputs and labels that is trained on labels as they arrive. Its label negatives are drawn within the batch, as in Li et al. Without labels, the input-only statistic cannot see a change in the label rule that leaves the inputs unchanged. We state this limit openly, and we test separately whether the label-conditional score block catches that case once delayed labels arrive.
Analysis. We will derive the statistic's sensitivity under a local alternative. In the standard local-alternative analysis, the detectable drift magnitude shrinks as $1/\sqrt{|B_t|}$ along directions the Fisher information "sees." Directions the parameter subset $S$ cannot represent are invisible to the test. This makes the choice of $S$ a measurable design axis rather than a hidden choice.
Baselines
Do nothing, and naive retraining. The do-nothing reference is the energy model and classifier frozen after the stationary prefix. The naive reference retrains on a sliding window at a fixed interval, with no detector. Both receive the same backbone and data.
Incremental Kolmogorov-Smirnov test. The first detection competitor is IKS (Fast Unsupervised Online Drift Detection Using Incremental Kolmogorov-Smirnov Test). It keeps reference and current windows in a balanced search tree, updates in $O(\log N)$ time per observation, and reports matching a fully labeled topline classifier while requesting only 42% to 47% of stream labels. It is the strongest cheap detector. We run it once per feature, with Bonferroni-corrected thresholds, on the same windows. We also reuse its label-request protocol, so that labels requested at a matched topline accuracy are directly comparable.
Maximum Concept Discrepancy. The learned multivariate competitor is MCD-DD (Online Drift Detection with Maximum Concept Discrepancy). It trains a contrastive encoder over sub-windows and flags drift when distances between sub-window embeddings exceed a dynamic threshold. It gets the same encoder capacity and tuning budget as our energy network.
Distribution-valued change detection. On FlowCAP-II, where each time step is a whole batch of cells, the reference is IDD (Beyond Euclidean Summaries). IDD monitors batches in 2-Wasserstein tangent space and reports an F1 of about 0.75 with a detection delay near one batch. Beating it means higher F1 at equal or lower delay.
Accuracy baselines. For prequential accuracy, the strongest baselines are AMRC and CURE. AMRC is reported best on 9 of 12 real streams at tens of milliseconds per step, but the file gives no per-dataset accuracies, so we rerun its released configuration on Electricity and NOAA. CURE (Bounded Context Management for Tabular Foundation Models) manages a 1,000-example context for a frozen tabular foundation model. It reports gains of +19.59 and +19.31 points over classical stream learners on RIALTO and METER, but no absolute accuracies appear in the file, so we rerun it as well. Beating CURE outright is not required. The decisive question is whether our method closes most of the gap at a comparable per-sample cost while also supplying calibrated detection. If CURE dominates on accuracy at lower cost, we will report that.
Ablations. Two ablations isolate the key ingredients, under an equal wall-clock budget per sample. The mean-energy monitor keeps the trained energy network but tracks only the scalar average energy of incoming windows. This is a one-dimensional projection of $g_t$ and needs no generator. The persistent-chain variant replaces the generator with warm-started Langevin chains at equal compute. A third ablation drops the covariance normalization, to show whether the Fisher weighting matters.
Why not just…?
Why not just monitor the average energy of incoming data? This is the most serious threat. For mean shifts and for points that move into high-energy regions, it may be enough. The argument against it is geometric. The scalar is blind to any drift that moves mass between regions of equal energy, such as one mode of a mixture swapping weight with another. The full score vector distinguishes which parameters the new data pull on, while the scalar does not. The mean-energy ablation on mode-reweighting streams and on copula streams settles this. If it matches the full statistic there, the generator is unnecessary for detection, and the contribution shrinks to adaptation alone.
Why not just use persistent Markov chains instead of a generator? Warm-started chains also track a slowly changing model. Under incremental drift they may be competitive. The expected failure is after abrupt drift, where chains must travel to new modes and mix slowly, while the generator re-fits in a few gradient steps. VERA's training-speed and stability results support this expectation in the static setting. The equal-compute persistent-chain ablation, scored by detection delay and recovery time after sudden drift on INSECTS and rotating-hyperplane streams, tests it directly.
Why not just run per-feature tests like IKS? IKS is far cheaper and has strong label-efficiency results. Per-feature tests, however, cannot see a change that preserves every marginal distribution, and the IDD paper shows that moment- and marginal-based monitors fail completely on such copula shifts. Our synthetic copula streams quantify this. On streams where drift is purely marginal, we expect IKS to tie or win on cost, and we will report that.
Why not just retrain a discriminative model, or use AMRC or CURE? These address accuracy, not calibrated label-free detection, and AMRC's kernel cost limits it in high dimensions. They remain the bar on accuracy, and the head-to-head comparisons above decide this question.
Proposed Experiments
Streams. The tabular and sensor streams are INSECTS (abrupt and incremental), Electricity, NOAA weather, Posture, METER, RIALTO, Anoshift, and FlowCAP-II. The synthetic streams are SEA, STAGGER, and rotating hyperplane, plus a family we construct with copula and mode-reweighting drift, where marginals are fixed by design. For images, we use a task-free class-incremental CIFAR-10 stream and CIFAR-100 on frozen features, comparable with the 30.28% replay-free result of Li et al.
Metrics. We report detection delay, false alarms per 10,000 points against the nominal rate, labels requested at matched topline accuracy, prequential accuracy, and per-sample latency.
Decisive checks. First, the full score statistic must beat both the mean-energy monitor and Bonferroni-corrected IKS on copula and reweighting drift, while holding its false-alarm rate near nominal on stationary segments. Second, at equal compute, the generator must recover faster than persistent chains after sudden drift. Third, the shared gradient used as the update must yield prequential accuracy competitive with AMRC on Electricity and NOAA.
Failing the first check reduces the idea to a cheaper energy-model trainer. Failing the second removes the case for amortization. Passing both would show that a minimax energy model can monitor and adapt itself in one computation.
References
- Adversarial concept drift detection under poisoning attacks for robust data stream mining
- Minimax Classification under Concept Drift with Multidimensional Adaptation and Performance Guarantees
- Energy-Based Models for Continual Learning
- No MCMC for me: Amortized sampling for fast and stable training of energy-based models
- Maximum Entropy Generators for Energy-Based Models.
- Fast Unsupervised Online Drift Detection Using Incremental Kolmogorov-Smirnov Test
- Online Drift Detection with Maximum Concept Discrepancy
- Beyond Euclidean Summaries: Online Change Point Detection for Distribution-Valued Data
- Bounded Context Management for Tabular Foundation Models on Stream Learning