An Automated Machine Learning Approach for Detecting Anomalous Peak Patterns in Time Series Data from a Research Watershed in the North Eastern United States Critical Zone

Ijaz Ul Haq, Byung Suk Lee, Donna M. Rizzo, Julia PerdrialSSRN 2023 · DOI

TL;DR

This paper introduces HF-PPAD, an automated machine learning framework designed to help hydrologists detect anomalous peak patterns in time series data from environmental sensors. By generating synthetic data and automating model selection, HF-PPAD makes advanced anomaly detection accessible to domain experts without deep ML knowledge.

What Problem Does This Paper Address?

Environmental sensors are increasingly used to monitor water quality, generating vast amounts of time series data. However, analyzing this data is often hampered by delays between data acquisition and analysis, largely due to the difficulty in rapidly identifying and cleaning anomalies. A particularly challenging type of anomaly in watershed data is "peak-pattern anomalies," which can arise from sensor malfunctions or natural events. Traditional anomaly detection methods often struggle with these pattern anomalies, requiring labeled data and expert knowledge to select appropriate models. This paper addresses the need for an automated, user-friendly solution that can effectively detect these peak patterns in watershed time series data.

Methods

The core of the HF-PPAD framework is an automated machine learning (AutoML) pipeline. It tackles the problem in two main stages:

  1. Synthetic Labeled Dataset Generation: To overcome the challenge of needing labeled data, HF-PPAD uses TimeGAN (Time-series Generative Adversarial Networks) to create a large synthetic dataset that mimics the characteristics of real watershed data. Crucially, it then injects synthetic "peak pattern anomalies" into this dataset, creating a fully labeled training set. This approach addresses data sparsity and skewness issues common in real-world hydrological data.
  2. Automated Best Model Instance Generation: HF-PPAD then employs an AutoML approach to find the best deep learning model for the task. It considers a pool of five state-of-the-art deep learning models: InceptionTime, MiniRocket, ResNet, TCN (Temporal Convolutional Network), and LSTM (Long Short-Term Memory). For each model, it automatically tunes architectural and training hyperparameters using optimizers like random forest, Bayesian optimization, Hyperband, and greedy search. The framework then allows users to specify a preference for accuracy versus model size (computational cost), outputting a recommended model instance that best balances these factors.

Datasets

The study utilizes a watershed time series dataset collected from a research watershed in Vermont. This dataset includes measurements of stream stage, turbidity, and fluorescent dissolved organic matter (FDOM) over a period of six years and four months. For training and validation, a synthetic dataset was generated using TimeGAN, comprising approximately 1 million time series samples. This synthetic dataset was then split into 70% for training and 30% for validation. The trained models were finally tested on a real test dataset.

Key Insights and Ideas

  • Synthetic Data as a Game-Changer: The ability to generate realistic synthetic data with injected anomalies is a critical enabler. It bypasses the expensive and time-consuming process of manual labeling, making supervised anomaly detection feasible for complex, real-world datasets.
  • AutoML for Domain Experts: HF-PPAD democratizes advanced anomaly detection by automating the complex model selection and hyperparameter tuning process. This empowers hydrologists, who may not have extensive ML expertise, to leverage powerful deep learning models.
  • Balancing Accuracy and Computational Cost: The framework explicitly addresses the inherent trade-off between model accuracy and computational resources. By allowing users to define their preference, it provides personalized recommendations, ensuring the chosen model is not only accurate but also practical for their specific needs.
  • Distinguishing Pattern Anomalies: The paper highlights the importance of detecting "pattern anomalies" (anomalies spanning multiple data points) over simpler "point anomalies," which is a key challenge in hydrological time series.

Key Results

  • High Detection Accuracy: The best-performing models achieved high accuracy for both FDOM and turbidity data, with balanced accuracy ranging from 70.2% to 97.3% and F-1 scores from 64.7% to 94.6%.
  • Synthetic Data Effectiveness: RNN regression models trained on synthetic data performed comparably to those trained on real data, demonstrating the suitability of TimeGAN-generated data for training ML models.
  • Model Performance Trade-offs:
    • InceptionTime and ResNet offered the highest accuracy for FDOM.
  • TCN and MiniRocket provided a good balance of accuracy and computational cost.
  • LSTM, while having the lowest accuracy, incurred the lowest computational cost, making it suitable for users prioritizing efficiency.
  • User Preference Adaptation: The framework successfully adapted model recommendations based on user-defined preferences for accuracy versus model size, as visualized in Figure 11.

Limitations and Future Work

  • Generalizability: The authors plan to test the framework on a wider range of WTSD and other environmental sensor data to validate its generalizability.
  • Model Pool Expansion: Future work includes incorporating additional machine learning models and expanding the search space for model generation.
  • Broader Applications: The framework's potential for other domains of anomalous events, such as those in water quality monitoring and flood forecasting, will be investigated.
  • NAP Anomaly Accuracy: The accuracy for "NAP" (not an anomalous peak type) was noted as lower, with some instances being mistaken for other anomaly types, suggesting room for refinement in distinguishing subtle differences.

Summary updated

BibTeX
@misc{haq2023automated,
  author = {Ijaz Ul Haq and Byung Suk Lee and Donna M. Rizzo and Julia Perdrial},
  title = {{An Automated Machine Learning Approach for Detecting Anomalous Peak Patterns in Time Series Data from a Research Watershed in the North Eastern United States Critical Zone}},
  year = {2023},
  doi = {10.2139/ssrn.4662943},
  howpublished = {SSRN},
}
Figure