Quantifying risky behavior in surgical simulation.

Christopher Sewell, Dan Morris, Nikolas H. Blevins, Federico Barbagli, Kenneth SalisburyPubMed 2005

TL;DR

This paper introduces a novel methodology for quantifying "risky" stylistic behaviors in surgical simulations, specifically for a mastoidectomy. By learning expert surgeon patterns for safe force, velocity, and visibility, the system provides trainees with detailed, visualized feedback to improve their technique beyond just avoiding critical errors.

What Problem Does This Paper Address?

Traditional surgical simulators often focus on whether a trainee successfully completes a procedure and avoids catastrophic errors (like damaging a nerve). However, they often fall short in teaching the more nuanced, "stylistic" behaviors that expert surgeons employ to minimize overall patient risk. These stylistic elements, such as maintaining optimal visibility and applying forces/velocities appropriate to proximity to vulnerable structures, are difficult to explicitly program but crucial for safe and efficient surgery. This paper tackles the challenge of quantifying and providing feedback on these subtle, yet important, risky behaviors in a simulated environment.

Methods

The core of the paper's methodology lies in learning "safe" operational envelopes from expert demonstrations. For a simulated mastoidectomy, they focus on three key risky behaviors:

  • Drill Force: They model the maximum safe force a trainee can apply as a function of the drill's distance from vulnerable structures (e.g., facial nerve, sigmoid sinus). This function is learned by recording expert surgeon forces and distances to these structures during a simulated procedure. They then fit regression models (linear and logistic) to the upper bounds of these force-distance relationships.
  • Drill Velocity: Similar to force, they learn a function for maximum safe drill velocity based on distance to vulnerable structures. This is also derived from expert demonstrations and fitted using regression techniques.
  • Visibility: They implement an algorithm to check if the bone being drilled is within the surgeon's field of view. This involves tracing a line from the drilled voxel to the camera's viewpoint and checking for obstructions. Performance is scored based on the percentage of hidden voxels removed.

The system then compares a trainee's actions against these learned expert models, providing both quantitative scores and qualitative, visual feedback (e.g., coloring regions of excessive force/velocity or poor visibility).

Datasets

The paper describes using data collected from an expert otolaryngologist performing a mastoidectomy procedure on their simulator. This data includes:

  • Voxel locations and applied force magnitudes during drilling.
  • Voxel locations and instantaneous drill velocity magnitudes.
  • Distances from each drilled voxel to six key vulnerable structures (sigmoid sinus, facial nerve, dura, incus, chorda tympani nerve, inner ear).

This data serves as the basis for learning the "safe" operational envelopes. No specific external datasets are mentioned; the data is generated from their own simulation environment.

Key Insights and Ideas

  • Beyond "Don't Damage": The paper highlights that surgical skill goes beyond simply avoiding direct injury. Stylistic behaviors that minimize potential risk are equally important and can be quantified.
  • Learning from Experts: "Programming by demonstration" is a powerful paradigm for capturing subtle surgical nuances that are hard to hardcode.
  • Context-Aware Safety: Safe operating parameters (force, velocity) are not absolute but are highly dependent on the proximity to critical anatomical structures.
  • Visual Feedback is Crucial: Simply providing a score isn't as effective as visualizing where and how a trainee made mistakes, allowing for targeted improvement.

Key Results

The paper demonstrates the feasibility of their methodology by:

  • Developing learned models for safe drill force and velocity as functions of distance to vulnerable structures.
  • Implementing a visibility check that identifies drilling of hidden bone.
  • Providing quantitative scoring for deviations from expert behavior in these three areas.
  • Enabling interactive 3D visualization where regions of risky behavior are highlighted with color, allowing trainees to pinpoint their mistakes. For example, Figure 4 shows a trainee's performance with visible voxels in green and hidden/risky voxels in red, with specific structures highlighted for context.

While specific quantitative performance improvements of trainees are not detailed, the paper establishes the framework for such evaluation.

Limitations and Future Work

  • Simulator Realism: The performance of the system is highly dependent on the visual and haptic realism of the simulator. As the simulator improves, expert profiles will need to be re-recorded.
  • Generalizability: While the principles of maintaining visibility and safe forces/velocities are common across surgery, the specific models are tuned for mastoidectomy. Extending this to other procedures will require adaptation.
  • Sophistication of Models: The authors hope to employ more sophisticated learning algorithms to better model expert profiles.
  • Additional Risky Behaviors: They aim to explore other risky surgical behaviors beyond the three addressed in this paper.
  • Graded Scoring for Visibility: Currently, visibility is binary (visible/hidden). The authors suggest that severity of consequences can vary by location, and a graded scoring/shading system based on distance to structures would be beneficial, though not yet implemented.

Summary updated

Abstract

Evaluating a trainee's performance on a simulated procedure involves determining whether a specified objective was met while avoiding certain "injurious" actions that damage vulnerable structures. However, it is also important to teach the stylistic behaviors that minimize overall risk to the patient, even though these criteria may be more difficult to explicitly specify and detect. In this paper, we address the development of metrics that evaluate the risk in a trainee's behavior while performing a simulated mastoidectomy. Specifically, we measure the trainee's ability to maintain an appropriate field of view so as to avoid drilling bone that is hidden from view, as well as to consistently apply appropriate forces and velocities. Models of the maximum safe force and velocity magnitudes as functions of distances from key vulnerable structures are learned from model procedures performed by an expert surgeon on the simulator. In addition to quantitatively scoring the trainee's performance, these metrics allow for interactive 3D visualization of the performance by distinctive coloring of regions in which excessive forces or velocities were applied or insufficient visibility was maintained, enabling the trainee to pinpoint his/her mistakes and how to correct them. Although these risky behaviors relate to a mastoidectomy simulator, the objectives of maintaining visibility and applying safe forces and velocities are common in surgery, so it may be possible to extend much of this methodology to other procedures.
BibTeX
@inproceedings{sewell2005quantifying,
  author = {Christopher Sewell and Dan Morris and Nikolas H. Blevins and Federico Barbagli and Kenneth Salisbury},
  title = {{Quantifying risky behavior in surgical simulation.}},
  booktitle = {PubMed},
  year = {2005},
}
Figure