ViSTR-GP: Online Cyberattack Detection via Vision-to-State Tensor Regression and Gaussian Processes in Automated Robotic Operations

Navid Aftabi, Philip Samaha, Jin Ma, Cheng Long, Ramy Harik, Dan Li

TL;DR

Industrial robots are highly vulnerable to "replay attacks," where hackers alter a robot's physical movements while feeding fake, normal-looking sensor data back to the monitoring system. This paper introduces ViSTR-GP, a framework that uses an independent overhead camera to visually estimate the robot's true joint angles and cross-checks them against the reported data. By combining lightweight tensor regression and Gaussian processes, it can detect physical deviations as small as 0.2 cm in real-time without needing complex new hardware.

What Problem Does This Paper Address?

As manufacturing moves toward Industry 4.0, industrial robots are increasingly connected to networks, making them prime targets for cyberattacks. One of the most dangerous threats is a data-integrity or replay attack. In this scenario, an attacker compromises the robot's controller to subtly alter its physical trajectory—potentially causing manufacturing defects or safety hazards. To avoid detection, the attacker intercepts the real-time sensor data and "replays" historical, normal data back to the operators.

Because the monitoring system relies entirely on the compromised internal network, traditional intrusion detection and physics-based models are easily fooled. While some researchers use side-channels like power consumption to verify robot behavior, these require intrusive hardware installations and are sensitive to factory noise. The authors needed a way to establish an unhackable "ground truth" of the robot's physical state using non-intrusive, off-the-shelf equipment.

Methods

The authors propose ViSTR-GP, a model-free, data-driven pipeline that uses a standard overhead camera as an independent side-channel. The camera sits completely outside the robot's control authority. The pipeline has three main steps:

  1. Vision-Based State Estimation: Incoming camera frames are processed by SAM-Track (a combination of the Segment Anything Model and a video tracker) to isolate the robot's silhouette from background factory clutter. This only requires a one-time manual click to initialize.
  2. Tensor Regression (TR): Instead of using a heavy, data-hungry Convolutional Neural Network (CNN), the authors use a low-rank bilinear tensor regression to map the 2D video mask directly to the robot's multi-joint angles.
  3. Residual Modeling via MVGP: The system calculates the "residual"—the difference between the camera's estimated joint angles and the (potentially hacked) encoder-reported angles. During normal operations, these residuals are modeled using a Matrix-Variate Gaussian Process (MVGP). The MVGP captures both the temporal structure (how noise changes throughout a movement cycle) and cross-joint correlations.
  4. Online Detection: In real-time, the system uses a Mahalanobis distance metric (a chi-squared test) to compare incoming residuals against the MVGP's predictive distribution. If the physical motion and the reported data diverge, the system flags an anomaly.

Datasets

The authors collected a custom dataset using a real-world robotic testbed at the Future Factories Laboratory (University of South Carolina).

  • Hardware: A Yaskawa GP8 robotic arm, a Siemens PLC, and an Intel RealSense L515 camera capturing video at 30 FPS.
  • Data: Synchronized video frames and encoder joint angles.
  • Splits: The model was trained purely on 10 nominal (attack-free) assembly cycles. It was evaluated on 4 custom replay attack scenarios featuring graded physical deviations at the robot's end-effector: 5 cm, 1 cm, 0.5 cm, and a highly subtle 0.2 cm.

Key Insights and Ideas

  • Vision as an unhackable side-channel: By placing a camera completely outside the robot's control network, attackers cannot spoof the physical ground truth without physically tampering with the camera lens.
  • Simpler math beats deep learning here: Surprisingly, the low-rank tensor regression (TR) significantly outperformed a deep CNN in estimating joint angles from masks. The TR model aligns better with the spatial structure of planar robot motion, reducing bias and variance.
  • Time-adaptive uncertainty is the secret sauce: Modeling the temporal correlation of the residuals (MVGP) rather than treating them as independent data points (IID) creates a much tighter, time-adaptive uncertainty band. If you ignore time, your baseline uncertainty has to be wide to account for the whole cycle, which allows subtle attacks to slip through undetected.

Key Results

  • Superior State Estimation: The TR model achieved a Root Mean Squared Error (RMSE) of 2.79° and a Mean Absolute Error (MAE) of 2.09° across all joints. This reduced error by ~55% compared to the CNN baseline.
  • Rapid Detection of Subtle Attacks: ViSTR-GP successfully detected all attack severities. Its advantage was most obvious in the hardest scenario (0.2 cm deviation). ViSTR-GP raised frequent alarms almost immediately after the attack started.
  • Baseline Comparisons: Ablation baselines (like CNN+IID or TR+IID) either suffered from massive detection delays or missed the subtle attacks almost entirely. The CNN's higher estimation bias inflated the residual noise, forcing the detector to use wider thresholds that masked the 0.2 cm attacks.

Limitations and Future Work

Limitations: * The system requires a fixed, secure camera viewpoint and stable calibration. If the camera is bumped, it requires recalibration. * Severe visual occlusions (e.g., a worker standing in front of the camera) will degrade the state estimation accuracy. * The current study evaluates a single robot family performing a highly repetitive task.

Future Work: The authors suggest fusing depth data (RGB-D) with the masks to improve robustness against lighting changes and occlusions. They also plan to incorporate lightweight kinematic priors (like link-length constraints) into the model, adopt state-space GPs for even faster inference, and expand the framework to monitor multiple robots simultaneously.

Summary updated

Abstract

Industrial robotic systems are central to automating smart manufacturing operations. Connected and automated factories face growing cybersecurity risks that can potentially cause interruptions and damages to physical operations. Among these attacks, data-integrity attacks often involve sophisticated exploitation of vulnerabilities that enable an attacker to access and manipulate the operational data and are hence difficult to detect with only existing intrusion detection or model-based detection. This paper addresses the challenges in utilizing existing side-channels to detect data-integrity attacks in robotic manufacturing processes by developing an online detection framework, ViSTR-GP, that cross-checks encoder-reported measurements against a vision-based estimate from an overhead camera outside the controller's authority. In this framework, a one-time interactive segmentation initializes SAM-Track to generate per-frame masks. A low-rank tensor-regression surrogate maps each mask to measurements, while a matrix-variate Gaussian process models nominal residuals, capturing temporal structure and cross-joint correlations. A frame-wise test statistic derived from the predictive distribution provides an online detector with interpretable thresholds. We validate the framework on a real-world robotic testbed with synchronized video frame and encoder data, collecting multiple nominal cycles and constructing replay attack scenarios with graded end-effector deviations. Results on the testbed indicate that the proposed framework recovers joint angles accurately and detects data-integrity attacks earlier with more frequent alarms than all baselines. These improvements are most evident in the most subtle attacks. These results show that plants can detect data-integrity attacks by adding an independent physical channel, bypassing the controller's authority, without needing complex instrumentation.
BibTeX
@misc{aftabi2025vistr,
  author = {Navid Aftabi and Philip Samaha and Jin Ma and Cheng Long and Ramy Harik and Dan Li},
  title = {{ViSTR-GP: Online Cyberattack Detection via Vision-to-State Tensor Regression and Gaussian Processes in Automated Robotic Operations}},
  year = {2025},
  eprint = {2509.10948},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.10948},
}
Figure