Developing an Unsupervised Real-Time Anomaly Detection Scheme for Time Series With Multi-Seasonality

Wentai Wu, Ligang He, Weiwei Lin, Yi Su, Yuhua Cui, Carsten Maple, Stephen A. Jarvis

Abstract

This paper presents AD-LTI, an unsupervised framework designed for real-time anomaly detection in complex, multi-seasonal time series. By integrating automated seasonal decomposition with a Gated Recurrent Unit (GRU)-based ensemble forecasting approach and a novel "Local Trend Inconsistency" (LTI) metric, the framework achieves state-of-the-art accuracy while maintaining the low latency required for production environments. AD-LTI is particularly robust against noise and "polluted" training data, common in unsupervised settings.

Introduction

In modern hyperconnected scenarios—such as data center monitoring, IoT sensor networks, and autonomous driving—time series data is massive, multi-channel, and rarely labeled. Detecting anomalies in these streams faces three major hurdles:

  1. Lack of Labels: Supervised learning is often impossible due to the high cost of manual annotation.
  2. Complex Seasonality: Signals frequently exhibit overlapping patterns (e.g., daily spikes atop weekly cycles), which traditional models struggle to untangle.
  3. Real-time Constraints: Detection must be fast enough for instant alerts yet robust enough to ignore noise or anomalies that may have leaked into the training set.

Existing prediction-driven methods often rely on simple frame-to-frame error, which can be unreliable if the "source" frame used for the prediction was itself an anomaly. AD-LTI addresses these issues through a consensus-based approach.

Methodology

1. The Backbone Model (GRU + ST)

Instead of requiring a Recurrent Neural Network (RNN) to learn complex seasonality from scratch, AD-LTI utilizes Prophet to explicitly extract seasonal features (daily and weekly terms). These are concatenated with raw features and fed into a Stacked Gated Recurrent Unit (GRU) network. This feature engineering approach reduces training time and improves accuracy by allowing the network to focus on residuals and local trends rather than long-term periodicities.

Overview of the AD-LTI framework architecture, showing the backbone GRU model with seasonal decomposition and the LTI-based anomaly scoring pipeline.

2. Local Trend Inconsistency (LTI)

The core innovation of this work is the Local Trend Inconsistency (LTI) metric. Rather than comparing the current data point to a single prediction, LTI evaluates an ensemble of forecasts.

As each new frame arrives, the model predicts a sequence of future frames. When the "future" actually arrives, the system compares the actual data against multiple historical predictions made at different prior time steps. Crucially, each prediction is weighted by the "reliability" of its source. If a prediction was based on a frame that was likely an anomaly, its influence in the ensemble is diminished, preventing error propagation.

Illustration of the Local Trend Inconsistency (LTI) calculation, comparing the actual frame at t3 with multiple historical predictions from t0, t1, and t2.

3. Automated Scoring

The LTI value is mapped to an Anomaly Score (AS) between 0 and 1 using a logistic-shaped function. The authors provide an iterative algorithm to automatically tune the parameters of this function, ensuring the system is "plug-and-play" for various applications without manual thresholding.

Experimental Evaluation

The model was evaluated on three distinct datasets: CalIt2 (building traffic), Dodgers Loop Sensor (freeway traffic), and a private Server Log dataset (multi-channel cybersecurity events). The Server Log dataset is particularly challenging due to high noise and complex inter-channel seasonality.

Key Results

  • Superior Accuracy: AD-LTI outperformed representative algorithms like OCSVM, Isolation Forest, and LSTM-AD. On the complex Server Log dataset, it achieved an AUC of 0.977, while most baseline models struggled to exceed 0.80.
  • Efficiency: Formulated using matrix operations, the LTI calculation allows for massive parallelization. Detection overhead was minimal, ranging from 0.18ms to 0.7ms per frame, making it ideal for high-frequency real-time streams.
  • Robustness: The model maintained high performance even when the prediction span was varied, demonstrating stability across different temporal windows.

Discussion and Conclusion

AD-LTI demonstrates that explicitly handling seasonality and using consensus-based detection can significantly improve unsupervised anomaly detection. By treating anomalies as "broken promises" across multiple historical perspectives, the model effectively filters out noise and self-corrects for past anomalies.

Future work will focus on extending the framework to asynchronized streams from disparate sensors and adapting the logic for large-scale distributed systems where computation may be split between edge devices and the cloud.

Summary updated

Abstract

On-line detection of anomalies in time series is a key technique used in various event-sensitive scenarios such as robotic system monitoring, smart sensor networks and data center security. However, the increasing diversity of data sources and the variety of demands make this task more challenging than ever. First, the rapid increase in unlabeled data means supervised learning is becoming less suitable in many cases. Second, a large portion of time series data have complex seasonality features. Third, on-line anomaly detection needs to be fast and reliable. In light of this, we have developed a prediction-driven, unsupervised anomaly detection scheme, which adopts a backbone model combining the decomposition and the inference of time series data. Further, we propose a novel metric, Local Trend Inconsistency (LTI), and an efficient detection algorithm that computes LTI in a real-time manner and scores each data point robustly in terms of its probability of being anomalous. We have conducted extensive experimentation to evaluate our algorithm with several datasets from both public repositories and production environments. The experimental results show that our scheme outperforms existing representative anomaly detection algorithms in terms of the commonly used metric, Area Under Curve (AUC), while achieving the desired efficiency.
BibTeX
@article{wu2022developing,
  author = {Wentai Wu and Ligang He and Weiwei Lin and Yi Su and Yuhua Cui and Carsten Maple and Stephen A. Jarvis},
  title = {{Developing an Unsupervised Real-Time Anomaly Detection Scheme for Time Series With Multi-Seasonality}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2022},
  doi = {10.1109/tkde.2020.3035685},
}
Figure