Interpretable Cyber-Physical Attack Detection in Smart Grids: A Leakage-Controlled Multi-Testbed Evaluation of Fusion, Robustness, and Observability Limits

Muhammad Ridwanul Hoque, Sarah Sharif, Reza Saeed Kandezy, Yaser BanadarXiv 2026 · arXiv

TL;DR

This paper exposes how standard evaluation practices in smart grid security often lead to "optimistic" performance claims by failing to account for temporal leakage and adversarial adaptivity. By introducing a transparent, cyber-physical fusion detector and a novel rate-invariant manifold test, the authors provide a more realistic—and sobering—assessment of our ability to detect stealthy grid attacks.

What Problem Does This Paper Address?

The modern electric grid is a "cyber-physical" system, blending digital control networks with physical power hardware. While this enables smarter management, it opens a massive attack surface. Researchers have proposed numerous machine learning (ML) detectors to catch intruders, but there's a catch: most are validated on "clean" data that has been randomly shuffled.

In a time-series environment like a power grid, shuffling is a cardinal sin. It allows a model to "see" into the future or past of a specific event, effectively memorizing patterns rather than learning to detect anomalies. Furthermore, many detectors are "black boxes," offering no explanation for their alarms, and they often ignore the physical constraints of the grid, making them vulnerable to "stealthy" False Data Injection (FDI) attacks that look perfectly normal to traditional monitoring tools.

Methods

The authors tackle these gaps with a multi-pronged approach:

  1. Interpretable Fusion: They built a Random Forest detector that fuses cyber (network traffic) and physical (grid measurements) features. Crucially, they use SHAP (SHapley Additive exPlanations) values to explain which features triggered an alarm, ensuring operators can trust the system's reasoning.
  2. Leakage-Controlled Evaluation: They moved beyond simple shuffling, testing their model against "rolling-origin" and "chronological hold-out" protocols that respect the arrow of time and prevent "leakage" from future data.
  3. Physics-Manifold Testing: To catch "slow-ramp" attacks—where an attacker slowly nudges grid values to avoid triggering temporal alarms—they developed a spatial consistency test. This test learns the "manifold" (the normal operating subspace) of the grid using Principal Component Analysis (PCA) and flags any state that drifts too far off this path, regardless of how slowly it got there.
  4. Adversarial Stress-Testing: They employed evolution strategies to find the most "stealthy" physics-feasible attacks possible to see where the detector truly breaks under pressure from an intelligent adversary.

Datasets

The study is impressively broad, utilizing three distinct data sources:

  • Dataset 1: A public paired cyber-physical dataset (SmartGrid-CyberPhysical-Attack-Dataset) for initial fusion testing.
  • IEEE 14-Bus & 118-Bus Testbeds: Simulated power systems used to generate physically realistic and "stealthy" FDI attacks that respect power-flow constraints.
  • Dataset 3: Real-world industrial control traffic (IEC 60870-5-104) to establish the limits of what can be detected from the network layer alone.

Key Insights and Ideas

  • Shuffling is a Mirage: Shuffling isn't just a minor error; it's a performance inflator. The authors found that a model's perceived accuracy can drop by over 15% simply by switching to a proper chronological evaluation.
  • The "Semantic" Blind Spot: Network headers are great for catching "structural" attacks (like someone replaying old commands), but they are totally blind to "semantic" attacks where an attacker changes the value inside a legitimate-looking packet. You must look at the physical measurements to catch these.
  • Interpretability is the 'Why': By using SHAP values, the authors show that both cyber features (like packet counts) and physical features (like current variance) are vital. This transparency is critical for grid operators who need to know why an alarm is sounding before they take drastic action.
Figure 1: SHAP importances: both cyber (green) and physical (blue) features matter.
  • Slow Ramps vs. Manifolds: Temporal detectors are like motion sensors—they catch sudden movements. A "slow ramp" attack is like a burglar moving at an inch per hour; the sensor eventually accepts the new position as the "new normal." The manifold test, however, is like a map—it knows where things should be, no matter how they moved.

Key Results

The quantitative results are a wake-up call for the field:

  • Protocol Gap: On the public dataset, macro-F1 scores plummeted from a near-perfect 0.96 under shuffling to a more realistic 0.80 under a strict temporal hold-out.
  • Adversarial Collapse: When faced with an adaptive attacker who knows the detector's weaknesses, performance crashed from 0.97 to as low as 0.33.
  • Manifold Superiority: In the challenging IEEE 118-Bus testbed, the temporal test began to fail at slow ramp rates (0.05°/step), while the manifold test maintained a 1.00 detection rate. At even slower rates (0.02°/step), the manifold test was still 3.4x more effective than the temporal one.
Figure 2: Evaluation protocol vs. leakage: shuffled CV is optimistic; time- blocked CV removes adjacency leakage; a temporal hold-out exposes distribu- tion shift.

Limitations and Future Work

The authors acknowledge that their study is "piecewise": they haven't yet fused cyber and physical data on a single, large-scale real system with a known topology. Their manifold test currently uses linear PCA, which might struggle with the highly non-linear dynamics of a grid under extreme stress or high renewable penetration.

Future work includes scaling these physics-feasible attacks to even larger systems, exploring "moving-target" defenses to keep attackers guessing, and refining non-linear manifold detection techniques.

Summary updated

Abstract

Cyber-physical smart-grid detectors are typically treated as black boxes and validated exclusively on clean, temporally shuffled datasets, which risk overly optimistic performance claims. This work presents a multi-testbed evaluation study built around an interpretable, time-windowed fusion Random Forest, adopting the pessimistic result whenever evaluation protocols disagree. (i) On a public paired cyber-physical dataset, shuffled cross-validation is optimistic: macro-F1 falls from 0.96 to 0.88 +/- 0.06 under rolling-origin temporal evaluation. A topology-free graph convolutional network (GCN) serving as a control confirms that this lightweight model is competitive on these splits. (ii) The adaptive and adaptive-joint feature-space attacks, which are infeasible as realistic grid manipulations, degrade macro-F1 to 0.67 and 0.34, respectively. Adversarial training and randomized smoothing partially restore performance, and a physics-constrained evolution-strategy evasion on the IEEE 14-Bus testbed shows the same vulnerability pattern. (iii) On real IEC-104 traffic, the cyber view reliably catches structural attacks (recall of 0.87 to 0.98) but largely fails on semantic value manipulation (recall of 0.11 to 0.35). An observability taxonomy on the IEEE 14-Bus system reveals that stealthy false data injection defeats all three observation views. (iv) On a full AC weighted least-squares state-estimation testbed, a forecasting-aided temporal test is evaded by slow ramps precisely where a rate-invariant physics-manifold test sustains detection at a common 5% false-alarm target. A full end-to-end evaluation on IEEE 118-Bus confirms the ordering but bounds its strength: the manifold test detects every trial down to 0.05 degree/step and 0.48 at 0.02 degree/step, against 0.14 for the temporal test. All code and benchmarks are released.
BibTeX
@misc{hoque2026interpretable,
  author = {Muhammad Ridwanul Hoque and Sarah Sharif and Reza Saeed Kandezy and Yaser Banad},
  title = {{Interpretable Cyber-Physical Attack Detection in Smart Grids: A Leakage-Controlled Multi-Testbed Evaluation of Fusion, Robustness, and Observability Limits}},
  year = {2026},
  eprint = {2611.10950},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10950},
}
Figure