An Adversarial Approach to Evaluating the Robustness of Event Identification Models

Obai Bahwal, Oliver Kosut, Lalitha Sankar

Abstract

Intelligent machine learning approaches are finding active use for event detection and identification that allow real-time situational awareness. Yet, such machine learning algorithms have been shown to be susceptible to adversarial attacks on the incoming telemetry data. This paper considers a physics-based modal decomposition method to extract features for event classification and focuses on interpretable classifiers including logistic regression and gradient boosting to distinguish two types of events: load loss and generation loss. The resulting classifiers are then tested against an adversarial algorithm to evaluate their robustness. The adversarial attack is tested in two settings: the white box setting, wherein the attacker knows exactly the classification model; and the gray box setting, wherein the attacker has access to historical data from the same network as was used to train the classifier, but does not know the classification model. Thorough experiments on the synthetic South Carolina 500-bus system highlight that a relatively simpler model such as logistic regression is more susceptible to adversarial attacks than gradient boosting.

Introduction

As modern power grids integrate increasing levels of renewable energy and distributed resources, operational complexity grows. Grid operators increasingly rely on Machine Learning (ML) for real-time situational awareness, utilizing high-fidelity data from Phasor Measurement Units (PMUs) to identify critical events such as sudden generation loss (GL) or load loss (LL).

In the power systems community, there is a strong preference for "interpretable" models combined with physics-based feature extraction over "black-box" deep learning architectures. However, the security of these specific pipelines against malicious actors has not been sufficiently scrutinized. This research addresses a critical vulnerability: are physics-aware, interpretable ML models robust against cyber-adversaries attempting to spoof grid events?

Methodology

Physics-Based Feature Extraction

Rather than feeding raw PMU data directly into a classifier, the authors utilize modal decomposition. This technique extracts the physical characteristics of the grid's dynamic response, focusing on features such as frequency, damping ratios, and residual coefficients. This ensures the model operates on features that have physical meaning within the context of power system dynamics.

Classification Models

Two standard, interpretable machine learning models are evaluated for their ability to distinguish between Generation Loss and Load Loss:

  1. Logistic Regression (LR): A linear model often favored for its simplicity and transparency.
  2. Gradient Boosting (GB): An ensemble method that builds a strong classifier from a collection of decision trees.

Adversarial Attack Framework

To evaluate robustness, the researchers developed an adversarial attack algorithm designed to flip the classifier's decision with minimal data tampering. The attack is evaluated in two scenarios:

  • White Box: The attacker possesses full knowledge of the trained classifier’s parameters.
  • Gray Box: The attacker understands the physics-based extraction method and has access to historical data from the network but does not know the specific model or parameters used by the operator.

The attack iteratively perturbs the extracted features in the direction of the classifier's negative gradient. Crucially, the algorithm then "reconstructs" the time-series signal using the inverse of the modal decomposition to ensure the resulting malicious data appears physically plausible to the PMU data stream.

Experimental Setup

The study utilizes the synthetic South Carolina 500-bus system, a standard benchmark for power systems research.

  • Data Generation: Using PSS/E (Power System Simulation for Engineering), the authors generated 1,500 event samples (750 GL and 750 LL).
  • Sensor Network: Data was collected from 95 PMUs strategically placed at the largest generator and load buses.
  • Telemetry: Each PMU provided three channels—voltage magnitude, voltage angle, and frequency—sampled at a rate of 30Hz.

Key Insights and Results

Baseline Performance

Under normal operating conditions without interference, both models performed exceptionally well. Gradient Boosting achieved near-perfect accuracy (AUC ≈ 1.0), while Logistic Regression achieved a strong AUC of approximately 0.90.

The Vulnerability of Interpretability

A primary finding of this work is that interpretability does not equate to security. Both LR and GB models proved highly susceptible to gradient-based attacks. In the white-box scenario, the system's accuracy plummeted to near zero when an attacker tampered with as few as 2% to 3% of the sensors (only 2 or 3 PMUs out of 95).

Model Resilience

While both models were vulnerable, Gradient Boosting demonstrated significantly higher robustness than Logistic Regression, particularly in the gray-box setting. This resilience is likely due to the ensemble nature of GB; its complex, non-linear decision boundaries are much harder for an attacker to approximate using a surrogate model compared to the linear boundaries of Logistic Regression.

Conclusion

This research demonstrates that physics-informed ML models, despite their interpretability, are highly vulnerable to targeted adversarial attacks. By manipulating a tiny fraction of the sensor network, an attacker can effectively blind a grid operator to the true nature of a system event.

Future research will focus on developing "adversarially robust" classifiers—models trained to recognize and ignore targeted perturbations—and building detection systems capable of distinguishing between legitimate physical disturbances and malicious data injections. As the grid moves toward automated response, securing the ML models that drive situational awareness is paramount.

Summary updated

Abstract

Intelligent machine learning approaches are finding active use for event detection and identification that allow real-time situational awareness. Yet, such machine learning algorithms have been shown to be susceptible to adversarial attacks on the incoming telemetry data. This paper considers a physics-based modal decomposition method to extract features for event classification and focuses on interpretable classifiers including logistic regression and gradient boosting to distinguish two types of events: load loss and generation loss. The resulting classifiers are then tested against an adversarial algorithm to evaluate their robustness. The adversarial attack is tested in two settings: the white box setting, wherein the attacker knows exactly the classification model; and the gray box setting, wherein the attacker has access to historical data from the same network as was used to train the classifier, but does not know the classification model. Thorough experiments on the synthetic South Carolina 500-bus system highlight that a relatively simpler model such as logistic regression is more susceptible to adversarial attacks than gradient boosting.
BibTeX
@inproceedings{bahwal2024adversarial,
  author = {Obai Bahwal and Oliver Kosut and Lalitha Sankar},
  title = {{An Adversarial Approach to Evaluating the Robustness of Event Identification Models}},
  booktitle = {SmartGridComm},
  year = {2024},
  url = {https://openreview.net/forum?id=C0OSFhEUpz},
}
Figure