A Framework for Checkpointing and Recovery of Hierarchical Cyber-Physical Systems

Kaustubh Sridhar, Radoslav Ivanov, Vuk Lesi, Marcio Juliato, Manoj R. Sastry, Lily L. Yang, James Weimer, Oleg Sokolsky, Insup LeearXiv 2022 · DOI · arXiv

Introduction

Cyber-physical systems (CPS), such as autonomous vehicles and medical robotics, are increasingly deployed in environments where sensor reliability is not guaranteed. Sensors are vulnerable to various anomalies, ranging from environmental noise (e.g., fog or glare) to sophisticated cyber-attacks like GPS spoofing. While robust state estimation techniques exist, many are computationally prohibitive for the resource-constrained microcontrollers typically found in embedded robotic systems. This paper presents a lightweight framework for checkpointing and recovery designed specifically for hierarchical, nonlinear CPS, enabling resilience without the overhead of complex optimization-based estimators.

System Architecture

The proposed framework introduces a structured approach to managing state consistency across multiple control layers. At its core is a Coordinator protocol that synchronizes state "saves" across the hierarchy. In a typical robot, an outer loop might handle slow-rate navigation (e.g., 10 Hz) while an inner loop manages high-rate motor control (e.g., 100 Hz). Without synchronization, recovering one layer from a past state while another continues with current (potentially corrupted) data leads to instability—a phenomenon known as the "domino effect."

The proposed hierarchical architecture featuring a Coordinator and integrated ADS, Recovery, and Checkpointing modules.

As shown in the architecture above, each sub-system is augmented with an Anomaly Detection System (ADS), a Recovery module, and a Checkpointing module. The Coordinator ensures that all layers capture their state at the same global time-instant, creating a "consistent" global checkpoint.

Roll-Forward Recovery (RFR)

When the ADS identifies a sensor anomaly, the system enters a recovery phase. Instead of relying on the compromised sensor data, the Roll-Forward Recovery (RFR) mechanism retrieves the most recent "clean" checkpoint. It then uses the history of saved control inputs to simulate the system's state forward to the current time.

Illustration of the Roll-Forward Recovery process across hierarchical layers with different sampling rates.

This process, illustrated above, allows the system to "ignore" the period of sensor corruption. Because real-world physics is nonlinear, the authors utilize Taylor series expansions to derive mathematical bounds on the maximum possible error accumulated during this open-loop simulation. This allows engineers to determine the "maximum tolerable anomaly duration"—the point at which the simulation error exceeds safety limits and requires a controlled system shutdown.

Evaluation and Key Results

The framework was validated through a numerical simulation of a differential drive ground robot. The hierarchy consisted of a 10 Hz trajectory controller (outer loop) and two 100 Hz DC motor controllers (inner loops). Under simulated sensor attacks (false data injection), the RFR framework maintained the robot's trajectory with high precision. In contrast, a standard Extended Kalman Filter (EKF) failed, as it attempted to incorporate the malicious sensor readings into its state estimate, leading to significant path deviation.

Key findings from the study include:

  • Efficiency: RFR only activates during detected anomalies, preserving CPU cycles during normal operation compared to continuous resilient estimators.
  • Theoretical Consistency: Experimental errors remained within the predicted theoretical bounds, validating the nonlinear error analysis.
  • Multi-rate Support: The Coordinator successfully managed the timing complexities of sub-systems operating at different frequencies.

Conclusion

This research provides a practical path toward making embedded CPS resilient to sensor failures and attacks. By focusing on consistent checkpointing and roll-forward simulation, the framework balances the need for safety with the realities of limited hardware resources. Future work will focus on extending this framework to distributed systems where network delays and packet loss introduce additional challenges to state synchronization.

Summary updated

Abstract

This paper tackles the problem of making complex resource-constrained cyber-physical systems (CPS) resilient to sensor anomalies. In particular, we present a framework for checkpointing and roll-forward recovery of state-estimates in nonlinear, hierarchical CPS with anomalous sensor data. We introduce three checkpointing paradigms for ensuring different levels of checkpointing consistency across the hierarchy. Our framework has algorithms implementing the consistent paradigm to perform accurate recovery in a time-efficient manner while managing the tradeoff with system resources and handling the interplay between diverse anomaly detection systems across the hierarchy. Further in this work, we detail bounds on the recovered state-estimate error, maximum tolerable anomaly duration and the accuracy-resource gap that results from the aforementioned tradeoff. We explore use-cases for our framework and evaluate it on a case study of a simulated ground robot to show that it scales to multiple hierarchies and performs better than an extended Kalman filter (EKF) that does not incorporate a checkpointing procedure during sensor anomalies. We conclude the work with a discussion on extending the proposed framework to distributed systems.
BibTeX
@misc{sridhar2022framework,
  author = {Kaustubh Sridhar and Radoslav Ivanov and Vuk Lesi and Marcio Juliato and Manoj R. Sastry and Lily L. Yang and James Weimer and Oleg Sokolsky and Insup Lee},
  title = {{A Framework for Checkpointing and Recovery of Hierarchical Cyber-Physical Systems}},
  year = {2022},
  eprint = {2205.08650},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.08650},
}
Figure