White Functionals for Anomaly Detection in Dynamical Systems

Marco Cuturi, Jean-Philippe Vert, Alexandre d'AspremontNeurIPS 2009 · OpenReview

Introduction

Detecting anomalies in real-world data is a complex challenge, primarily because most traditional anomaly detection methods are designed for independent and identically distributed (i.i.d.) data. However, real-world data—such as sensor readings on an engine or stock market ticks—often comes from dynamical systems where the current state is heavily dependent on the past.

The traditional approach to this problem is "predict-and-compare": build a model to predict the next state and flag an anomaly if the prediction error (the residue) is too high. This paper argues that this approach is often unnecessarily difficult. Instead of trying to predict the entire next state, the authors ask: Can we find a specific "view" or projection of the data that is naturally noisy and stationary, and then monitor if that specific view starts looking "non-random"?

To answer this, the paper introduces "white functionals"—mathematical functions that, when applied to a dynamical system's state, produce a signal with very low autocorrelation, resembling white noise. By identifying these functionals during normal system behavior, we can flag anomalies whenever the resulting signal deviates from this "whiteness," offering a robust way to detect subtle changes in complex, high-dimensional time series.

The "White Functional" Approach

The core methodology revolves around finding these white functionals within a Reproducing Kernel Hilbert Space (RKHS). These are non-linear functions $\alpha$ such that the sequence $\alpha(Z_t)$ has low autocorrelation and high variance.

To achieve this, the authors adapt two metrics from econometrics and signal processing: the Autocorrelation Factor (MAC) and Predictability (Box-Tiao). Minimizing these criteria helps find the "whitest" possible projections of the data. By utilizing kernel methods, the approach can uncover complex, non-linear relationships in high-dimensional data that linear methods (like standard PCA) would miss.

To prevent the model from picking up "junk" noise or finding degenerate solutions in infinite-dimensional spaces, the search is restricted to a subspace spanned by the top $p$ eigenfunctions of the covariance operator (Subspace Regularization). Additionally, Tikhonov regularization is employed to ensure the mathematical operators remain stable during estimation.

Experimental Validation: The Lotka-Volterra System

The authors evaluate their method using a simulated Lotka-Volterra population dynamics model. This 4-species competitive model is known to produce chaotic, non-linear behavior, making it an excellent testbed for dynamical anomaly detection.

To test the methods, 40 random perturbations were injected into trajectories of 800 time steps. These anomalies were designed as "unusual increments"—sudden jumps that did not follow the natural physics of the system. The model utilized a mixture kernel that considers both the current state and the most recent change (increment) in the system.

Lotka-Volterra System and Anomaly Detection Signals

As seen in the system trajectories, the proposed Box-Tiao and kMAC methods produce distinct spikes corresponding to the injected anomalies, whereas baseline methods struggle to isolate these events from the system's natural chaotic fluctuations.

Key Results and Insights

The authors compared their MAC and Box-Tiao (BT) functionals against standard benchmarks: One-Class SVM (ocSVM) and Kernel PCA (kPCA).

  • Superiority in Subtle Detection: When the anomalies were small (low noise amplitude $\sigma_\delta$), the white functionals significantly outperformed ocSVM and kPCA. For example, at low noise levels, the BT and MAC methods maintained an AUC (Area Under Curve) near 0.8, while ocSVM hovered near 0.5, which is equivalent to random guessing.
  • Robustness: As the anomalies became larger and more obvious, all methods eventually performed well. However, the white functionals were much more sensitive to the "difficult" cases where the anomaly was buried in the system's natural chaos.
  • Efficiency: The method provides a canonical decomposition of the Hilbert space, allowing researchers to rank eigenfunctions by their "whiteness" and use the most effective ones for monitoring.
AUC Performance vs. Noise Amplitude

Conceptually, this work flips the script on Slow Feature Analysis. While many researchers look for "slow features" (signals that change slowly over time) to understand systems, this paper looks for the "fastest," most random-looking features to detect when the underlying "physics" of the system breaks. It elegantly connects modern kernel methods with the classical econometric concept of cointegration—the idea that even if individual variables are erratic, certain combinations of them might be stable.

Limitations and Future Directions

While promising, the current formulation focuses on order-1 autocovariance (how state $t$ relates to state $t+1$). The authors note this could be generalized to higher-order lags to capture more complex temporal dependencies.

Furthermore, the method assumes the underlying "normal" process is second-order stationary. It might struggle with systems that have heavy seasonal trends or evolving baselines without prior preprocessing. Finally, the paper uses simple thresholding for the alarm scores; integrating more advanced statistical change-point detection algorithms, like CUSUM, could further improve flagging accuracy in real-time applications.

Summary updated

Abstract

We propose new methodologies to detect anomalies in discrete-time processes taking values in a set. The method is based on the inference of functionals whose evaluations on successive states visited by the process have low autocorrelations. Deviations from this behavior are used to flag anomalies. The candidate functionals are estimated in a subset of a reproducing kernel Hilbert space associated with the set where the process takes values. We provide experimental results which show that these techniques compare favorably with other algorithms.
BibTeX
@inproceedings{cuturi2009white,
  author = {Marco Cuturi and Jean-Philippe Vert and Alexandre d'Aspremont},
  title = {{White Functionals for Anomaly Detection in Dynamical Systems}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2009},
  url = {https://openreview.net/forum?id=BkErbF-_Zr},
}
Figure