Between Zeros and Ones: Behavioral Characterization Beyond Binary Labeling Across Public ICS Datasets

Konstantinos E. Kampourakis, Vyron Kampourakis, Georgios Spathoulas, Constantinos KoliasarXiv 2026 · DOI · arXiv

TL;DR

This paper introduces a novel framework to characterize cyber-physical attacks in Industrial Control Systems (ICS) by analyzing raw process data into five interpretable "behavioral primitives" (drift, spike, oscillation, repetition, switching). This approach reveals that attacks are not a monolithic "attack" class but exhibit diverse temporal patterns, highlighting limitations of current binary evaluation methods and suggesting a need for behavior-stratified benchmarking.

What Problem Does This Paper Address?

Intrusion detection in Industrial Control Systems (ICS) is crucial for maintaining the stability and safety of critical infrastructure. Current research and evaluation of ICS Intrusion Detection Systems (IDS) heavily rely on public datasets labeled with a simple binary distinction: "normal" or "attack." This binary labeling practice, while convenient, masks the rich diversity of how cyber-physical attacks can manifest. Real-world attacks can involve subtle sensor drifts, sudden spikes, frozen telemetry, or repetitive control actions, each requiring different responses from operators. By collapsing all these distinct attack behaviors into a single "attack" label, current evaluation methods can provide a misleadingly optimistic view of IDS performance and fail to expose blind spots in detecting specific attack types. This paper aims to address this by providing a more nuanced way to characterize and evaluate ICS attack behaviors.

Methods

The core of this paper is a novel behavioral characterization framework. This framework takes raw, multivariate time-series data from ICS processes and transforms it into a set of five interpretable behavioral primitives:

  • Drift: Measures sustained monotonic deviations and gradual process shifts.
  • Spike: Captures abrupt transient deviations.
  • Oscillation: Quantifies periodic fluctuations and directional reversals.
  • Repetition: Detects recurring temporal patterns.
  • Switching: Counts discrete state or mode changes.

These primitives are extracted from segmented, overlapping temporal windows of the process data. For each window, the framework computes a five-dimensional behavioral vector by aggregating the primitive scores across all process variables using the 95th percentile. This aggregation method balances capturing localized attack effects with filtering out isolated sensor glitches. To further simplify analysis, a dominant behavioral property is assigned to each window based on the highest primitive activation. This framework is designed to be observational, focusing on what is directly measurable in the data without requiring deep domain knowledge or inferring attacker intent.

Datasets

The authors evaluated their framework on three widely used public ICS benchmarks:

  • SWaT (Secure Water Treatment)
  • WADI (Water Distribution)
  • HAI (Hybrid-Assisted Industrial)

These datasets are chosen for their relevance to multivariate time-series anomaly detection in ICS. The paper does not introduce new datasets but focuses on re-evaluating existing ones through the lens of behavioral characterization.

Key Insights and Ideas

  • Attacks are not monolithic: The most significant insight is that "attack" is not a single, homogeneous class. Cyber-physical attacks manifest in diverse temporal patterns, making binary labels insufficient for comprehensive evaluation.
  • Dataset bias is real: Different ICS datasets emphasize different behavioral primitives. This means that an IDS performing well on one dataset might be biased towards specific attack types and perform poorly on others.
  • Binary evaluation hides critical information: Aggregate binary metrics can mask significant performance drops when evaluating specific behavioral proxies. This suggests that current benchmarks might not be as robust as they appear.
  • Behavioral primitives offer a richer understanding: The five proposed primitives provide a more interpretable and granular way to understand attack dynamics, moving beyond simple "attack" vs. "normal" labels.

Key Results

  • Behavioral shifts are evident: Across all three datasets (SWaT, WADI, HAI), attack windows consistently show measurable deviations from normal operation across the behavioral primitives.
  • Dataset-specific profiles:
    • HAI is characterized by high spike and drift activity.
  • SWaT shows increased oscillation and decreased repetition.
  • WADI exhibits smaller shifts, primarily in spike, with relatively stable oscillation.
  • Significant performance drop with behavior-proxy evaluation: When a Random Forest classifier was tasked with distinguishing between the dominant behavioral primitives (multiclass) instead of just normal vs. attack (binary), macro F1 scores dropped dramatically. For example, in SWaT, macro F1 fell from 85.44% to 37.84%. This highlights how binary evaluation can obscure performance variations across different attack behaviors.
  • High agreement with human annotation: The automated dominant label assignment showed a high agreement (98.3% observed, 0.978 Cohen's κ) with manual annotations by the authors, validating the interpretability of the framework's assignments.

Limitations and Future Work

Limitations:

  • Observational, not causal: The framework describes observable temporal deviations but does not infer attacker intent, semantic attack categories, or causal mechanisms.
  • Limited data scope: It only uses LO/L1 process data, meaning purely cyber attacks that don't affect physical variables are not captured.
  • Primitive selection: The five chosen primitives are a conservative baseline; other descriptors (e.g., spectral, entropy-based) could offer a richer characterization.
  • Fixed window size: The 60-second window might smooth short transients or truncate slow degradations.
  • Single dominant label: Assigning a single dominant label can hide mixed behaviors within a window.
  • Unitless morphology: Primitive values describe temporal morphology, not direct physical severity, making cross-dataset comparisons about shape, not impact.

Future Work:

  • Expand the behavioral vocabulary with richer descriptors.
  • Develop mixed-label or multi-label representations to better reflect co-occurring behaviors.
  • Evaluate across more datasets and ideally with expert-curated annotations.
  • Integrate behavioral characterization with IDS triage, alert prioritization, and incident response for operational utility.

Summary updated

Abstract

Intrusion detection in Industrial Control Systems (ICS) is typically evaluated on a small set of public benchmarks using binary ``normal'' versus ``attack'' labels, a practice that can mask the behavioral diversity of cyber-physical attacks. To address this limitation, we propose a behavioral characterization framework that maps raw multivariate process traces into five interpretable physical primitives: drift, spike, oscillation, repetition, and switching. We apply the framework to three widely used ICS benchmarks, namely, SWaT, WADI, and HAI, and show that attack windows exhibit clear behavioral shifts relative to normal operation while the three datasets occupy largely distinct regions of the behavioral space, revealing both cross-dataset bias and intra-dataset diversity. In particular, WADI is dominated by repetition, HAI emphasizes sustained drift and oscillation, and SWaT is characterized by stealthier frozen-telemetry behavior. To examine the evaluation implications, we use an indicative Random Forest baseline and show that aggregate binary metrics can limit visibility into performance across different behavioral proxies. For example, in SWaT, macro F1 drops from 85.44% under binary evaluation to 37.84% under behavior-proxy multiclass prediction, with similar degradations observed on WADI and HAI. Based on these findings, we argue for complementing conventional binary benchmarking with behavior-stratified evaluation to expose blind spots that aggregate scores leave hidden and to better support targeted incident response.
BibTeX
@misc{kampourakis2026between,
  author = {Konstantinos E. Kampourakis and Vyron Kampourakis and Georgios Spathoulas and Constantinos Kolias},
  title = {{Between Zeros and Ones: Behavioral Characterization Beyond Binary Labeling Across Public ICS Datasets}},
  year = {2026},
  eprint = {2606.30493},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.30493},
}

Loading related content...

Figure