TL;DR
This paper introduces a novel framework to characterize cyber-physical attacks in Industrial Control Systems (ICS) by analyzing raw process data into five interpretable "behavioral primitives" (drift, spike, oscillation, repetition, switching). This approach reveals that attacks are not a monolithic "attack" class but exhibit diverse temporal patterns, highlighting limitations of current binary evaluation methods and suggesting a need for behavior-stratified benchmarking.
What Problem Does This Paper Address?
Intrusion detection in Industrial Control Systems (ICS) is crucial for maintaining the stability and safety of critical infrastructure. Current research and evaluation of ICS Intrusion Detection Systems (IDS) heavily rely on public datasets labeled with a simple binary distinction: "normal" or "attack." This binary labeling practice, while convenient, masks the rich diversity of how cyber-physical attacks can manifest. Real-world attacks can involve subtle sensor drifts, sudden spikes, frozen telemetry, or repetitive control actions, each requiring different responses from operators. By collapsing all these distinct attack behaviors into a single "attack" label, current evaluation methods can provide a misleadingly optimistic view of IDS performance and fail to expose blind spots in detecting specific attack types. This paper aims to address this by providing a more nuanced way to characterize and evaluate ICS attack behaviors.
Methods
The core of this paper is a novel behavioral characterization framework. This framework takes raw, multivariate time-series data from ICS processes and transforms it into a set of five interpretable behavioral primitives:
- Drift: Measures sustained monotonic deviations and gradual process shifts.
- Spike: Captures abrupt transient deviations.
- Oscillation: Quantifies periodic fluctuations and directional reversals.
- Repetition: Detects recurring temporal patterns.
- Switching: Counts discrete state or mode changes.
These primitives are extracted from segmented, overlapping temporal windows of the process data. For each window, the framework computes a five-dimensional behavioral vector by aggregating the primitive scores across all process variables using the 95th percentile. This aggregation method balances capturing localized attack effects with filtering out isolated sensor glitches. To further simplify analysis, a dominant behavioral property is assigned to each window based on the highest primitive activation. This framework is designed to be observational, focusing on what is directly measurable in the data without requiring deep domain knowledge or inferring attacker intent.
Datasets
The authors evaluated their framework on three widely used public ICS benchmarks:
- SWaT (Secure Water Treatment)
- WADI (Water Distribution)
- HAI (Hybrid-Assisted Industrial)
These datasets are chosen for their relevance to multivariate time-series anomaly detection in ICS. The paper does not introduce new datasets but focuses on re-evaluating existing ones through the lens of behavioral characterization.
Key Insights and Ideas
- Attacks are not monolithic: The most significant insight is that "attack" is not a single, homogeneous class. Cyber-physical attacks manifest in diverse temporal patterns, making binary labels insufficient for comprehensive evaluation.
- Dataset bias is real: Different ICS datasets emphasize different behavioral primitives. This means that an IDS performing well on one dataset might be biased towards specific attack types and perform poorly on others.
- Binary evaluation hides critical information: Aggregate binary metrics can mask significant performance drops when evaluating specific behavioral proxies. This suggests that current benchmarks might not be as robust as they appear.
- Behavioral primitives offer a richer understanding: The five proposed primitives provide a more interpretable and granular way to understand attack dynamics, moving beyond simple "attack" vs. "normal" labels.
Key Results
- Behavioral shifts are evident: Across all three datasets (SWaT, WADI, HAI), attack windows consistently show measurable deviations from normal operation across the behavioral primitives.
- Dataset-specific profiles:
- HAI is characterized by high spike and drift activity.
- SWaT shows increased oscillation and decreased repetition.
- WADI exhibits smaller shifts, primarily in spike, with relatively stable oscillation.
- Significant performance drop with behavior-proxy evaluation: When a Random Forest classifier was tasked with distinguishing between the dominant behavioral primitives (multiclass) instead of just normal vs. attack (binary), macro F1 scores dropped dramatically. For example, in SWaT, macro F1 fell from 85.44% to 37.84%. This highlights how binary evaluation can obscure performance variations across different attack behaviors.
- High agreement with human annotation: The automated dominant label assignment showed a high agreement (98.3% observed, 0.978 Cohen's κ) with manual annotations by the authors, validating the interpretability of the framework's assignments.
Limitations and Future Work
Limitations:
- Observational, not causal: The framework describes observable temporal deviations but does not infer attacker intent, semantic attack categories, or causal mechanisms.
- Limited data scope: It only uses LO/L1 process data, meaning purely cyber attacks that don't affect physical variables are not captured.
- Primitive selection: The five chosen primitives are a conservative baseline; other descriptors (e.g., spectral, entropy-based) could offer a richer characterization.
- Fixed window size: The 60-second window might smooth short transients or truncate slow degradations.
- Single dominant label: Assigning a single dominant label can hide mixed behaviors within a window.
- Unitless morphology: Primitive values describe temporal morphology, not direct physical severity, making cross-dataset comparisons about shape, not impact.
Future Work:
- Expand the behavioral vocabulary with richer descriptors.
- Develop mixed-label or multi-label representations to better reflect co-occurring behaviors.
- Evaluate across more datasets and ideally with expert-curated annotations.
- Integrate behavioral characterization with IDS triage, alert prioritization, and incident response for operational utility.
Summary updated