Detection and Prediction of Adverse and Anomalous Events in Medical Robots

AAAI 2013

TL;DR

This paper introduces a system using dynamic Bayesian networks (DBNs) to detect and predict adverse and anomalous (A&A) events in medical robots. By modeling both hardware and software states, their approach achieves high accuracy in identifying and forecasting potential failures, enhancing surgical robot safety.

What Problem Does This Paper Address?

Medical robotic systems, while offering precision and minimally invasive capabilities, are inherently complex cyberphysical systems. This complexity, stemming from intricate electromechanical designs and sophisticated software, increases the risk of dangerous accidents due to hardware malfunctions, software errors, or human-machine interface issues. Existing safety measures often focus on design-time analysis (like FMEA or model checking), but ensuring complete safety in real-world, dynamic operating conditions remains a significant challenge. This paper tackles the critical need for robust, online monitoring systems that can not only detect A&A events as they occur but also predict them before they escalate, thereby improving patient safety and procedural success.

Methods

The core of the proposed system lies in using dynamic Bayesian networks (DBNs) to model the time-evolution of the robot's state. DBNs are probabilistic graphical models that represent the relationships between variables over time.

Here's a breakdown of their approach:

  • Data Collection: They collect data from both the hardware (e.g., joint angles, motor torques) and software (e.g., high-level actions, intermediate computation results) of a small animal biopsy robot (SABiR). This comprehensive data collection is crucial for capturing the full system dynamics.
  • DBN Modeling: They learn DBNs to model the "normal" state transitions of the robot. These models predict the probability of the next state given the current one.
  • Conditional Probability Distributions (CPDs): For variables that change linearly, they use linear Gaussian models. For more complex, non-linear dynamics, they employ regression trees. This combination allows for a flexible representation of the system's behavior.
  • Sparse Candidate Algorithm: To manage the complexity of learning DBNs, they use the "sparse candidate" algorithm to limit the number of parent variables considered for each node, making the learning process more efficient.
  • Simulation as an Oracle: A key aspect is the use of a Simulink-based simulator of the SABiR robot. This simulator acts as an "oracle," generating data to train and validate the DBNs, especially for rare A&A events, ensuring accurate model learning.
  • Detection and Prediction:
    • Detection: A&A events are detected by identifying deviations from the expected "normal" behavior modeled by the DBN, often signaled by a smoothed negative log-likelihood (NLL) score.
  • Prediction: To predict future A&A events, the DBN is used to generate a sequence of likely future states. The NLL score is then computed based on how much the predicted needle tip trajectory deviates from the intended reference trajectory.

Datasets

The paper primarily uses data generated from their Simulink-based simulator of the SABiR robot. This simulator is configured with a "gel block" to mimic tissue resistance. They generate sequences of:

  • Normal trajectories: Used to train the DBNs to model typical robot behavior.
  • A&A event trajectories: Specifically, they simulate three types of A&A events:
  • Encoder Failure: A simulated loss of position reporting from a motor.
  • Sweep Event: The needle tip grazing the tissue surface before insertion.
  • Out-of-Workspace (OOW) Event: A simulated software bug leading to an illegal target position.

The A&A data is intentionally kept at a low proportion (1.25%) in the evaluation datasets to reflect the rarity of such events in real-world scenarios.

Key Insights and Ideas

  • Joint Hardware-Software Modeling is Crucial: The study strongly suggests that modeling both hardware and software variables provides significant advantages in detecting and predicting A&A events, especially for certain types of anomalies. Software variables, like estimated forces or high-level action flags, capture nuances not available from hardware alone.
  • Simulation as a Powerful Training Tool: Using a simulator as an "oracle" to generate data for training DBNs is a highly effective strategy, particularly for learning from rare events like A&A scenarios. This circumvents the difficulty of collecting sufficient real-world failure data.
  • Predictive Power Enhances Safety: The ability to predict A&A events before they occur, even by a small margin (e.g., 0.05s to 1s), offers a critical window for intervention, significantly improving safety over mere detection.
  • DBNs are Versatile for Complex Dynamics: The combination of linear Gaussian models and regression trees within DBNs proves effective in capturing the complex, non-linear dynamics of a robotic system.

Key Results

The paper demonstrates the effectiveness of their DBN-based system across several metrics:

  • Accurate Normal Behavior Modeling: The DBNs achieve high accuracy (r² values around 0.95-1.00) in modeling the time-evolution of hardware variables. While predicting software variables is more challenging, the accuracy is sufficient for A&A event detection.
  • Rapid A&A Event Detection: At a false positive rate (FPR) of 0.06, their models can detect A&A events very quickly, often within milliseconds (e.g., 1-10ms for encoder failures). The HS10 model (which includes lookahead) shows particular strength in detecting sweep events.
  • Effective A&A Event Prediction: The HS10 model also excels at predicting sweep events, achieving earlier predictions (k=100, 1000, 2000 steps) compared to other models. The ability to predict events up to 1 second in advance is demonstrated.
  • Model Performance: The model that incorporates both hardware and software variables, and also includes a "lookahead" capability (HS10), generally performs best in both detection and prediction tasks.

Limitations and Future Work

  • Limitations:
    • The evaluation is performed using a simulator, and real-world performance may differ.
  • The study focuses on a specific small animal biopsy robot; generalization to other medical robots may require re-training or adaptation.
  • While they consider software variables, the full spectrum of potential software-induced anomalies might not be exhaustively covered.
  • Future Work:
    • Improving the simulation and controller by adding path planning and image-guidance capabilities.
  • Developing more complex high-level actions and collecting more software execution data.
  • Investigating an active-learning framework for more efficient model training.
  • Developing clinically effective response strategies to the detected/predicted A&A events.

Summary updated

BibTeX
@inproceedings{2013detection,
  title = {{Detection and Prediction of Adverse and Anomalous Events in Medical Robots}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2013},
}
Figure