Maxillofacial surgical simulation system with haptic feedback

Jing Zhang, Jiahui Qian, Han Zhang, Ling He, Bin Li, Jing Qin, Hong‐Ning Dai, Wei Tang, Weidong TianJournal of Industrial and Management Optimization 2020 · DOI

TL;DR

This paper presents a novel virtual simulation system for maxillofacial surgery that incorporates haptic feedback. The system aims to improve surgical training and planning by providing a realistic, interactive environment for practicing complex procedures like osteotomy and plate fixation, ultimately reducing training costs and enhancing skill acquisition.

What Problem Does This Paper Address?

Maxillofacial surgery is inherently complex, requiring extensive training for surgeons to achieve proficiency and minimize patient risk. Traditional training methods, such as operating room experience, cadaveric models, and rapid prototyping, have limitations including limited access, high costs, and anatomical discrepancies. While computer-aided surgery and 3D simulation have emerged, many existing systems lack crucial haptic feedback, which is vital for developing the tactile sense and fine motor control necessary for delicate surgical maneuvers. This paper addresses the need for a comprehensive virtual training platform that bridges this gap by integrating realistic visual and haptic feedback for maxillofacial surgical procedures.

Methods

The core of the system is an integrated-model architecture that combines visual, collision, and mechanical models. A six-degree-of-freedom (6-DoF) haptic device is used to provide force feedback to the user.

  • 3D Modeling: Anatomic structures and surgical instruments are reconstructed from CT and laser scanning data.
  • Collision Detection: An Axis-Aligned Bounding Box (AABB) algorithm is employed for efficient collision detection between surgical tools and bone structures.
  • Force Rendering: A two-layer mechanism model is designed to balance real-time response and force feedback accuracy, considering the complex mechanical properties of bone tissue. This model calculates output forces based on detected collisions and mechanical properties.
  • Haptic Feedback: A 1000Hz haptic rendering loop ensures realistic and stable force feedback. A quadratic second-order digital filter is used to smooth the force signals and prevent device vibration.
  • Simulation of Operations: The system simulates essential maxillofacial surgery operations, including:
  • Open Reduction: Realigning fractured bone segments.
  • Osteotomy: Cutting or reshaping bone tissue.
  • Plate Fixation: Designing and placing surgical plates to stabilize bone.
  • Grinding Function: A novel interactive grinding algorithm is implemented for bone reshaping, which involves processing model data, calculating collision detection, and reconstructing the mesh.

Datasets

Not discussed in the paper. The paper mentions using CT and laser scanning data for reconstructing 3D models of anatomic structures and surgical instruments.

Key Insights and Ideas

  • Haptic Feedback is Crucial for Maxillofacial Training: The paper strongly emphasizes that haptic feedback is not just a nice-to-have but a critical component for effective surgical training, enabling surgeons to develop a sense of touch and precision that visual-only simulations cannot replicate.
  • Two-Layer Mechanism for Bone Simulation: The novel two-layer mechanism model for bone tissue is a key innovation. It addresses the inherent challenge of balancing the need for real-time responsiveness in simulations with the accurate representation of complex bone mechanics, which is essential for realistic force feedback.
  • Integrated Approach for Comprehensive Simulation: The use of an integrated-model architecture, combining visual, collision, and mechanical models, allows for a holistic simulation of the surgical environment, from visual rendering to physical interaction.
  • Patient-Specific Guide Plate Design: The system's fast response capability allows for real-time design of patient-specific guide plates, a crucial aspect of modern maxillofacial surgery, enhancing planning accuracy.

Key Results

  • Realistic Simulation of Key Operations: The system successfully simulated open reduction, osteotomy, and plate fixation, providing users with haptic feedback during these procedures.
  • Improved Surgical Planning: The system assists surgeons in planning more accurate surgical interventions and designing patient-specific implants.
  • Positive User Evaluation: Ten stomatology surgeons evaluated the system, scoring it an average of eight out of ten across immersion, user-friendliness, stability, and surgical training effectiveness.
  • Performance Metrics:
    • Without grinding function, the system achieved high frame rates (e.g., 72.2 FPS, 145.8 FPS) with detailed models, ensuring real-time performance.
  • With the grinding function, the FPS decreased (e.g., 16.6 FPS, 30.0 FPS) due to increased computational load, but the paper suggests model optimization and simplification can mitigate this.

Limitations and Future Work

  • Soft Tissue Simulation: The authors acknowledge that the current system primarily focuses on bone tissue and that simulating soft tissues, fluids, and other organs remains a significant challenge for achieving a truly comprehensive surgical simulation.
  • Authenticity of Operation: While the bone simulation is precise, the authenticity of operations, particularly the visual realism of grinding, needs further improvement.
  • Technical Challenges: The paper notes that there are still many technical challenges in the development of virtual surgery, as real-world surgery is far more complex than current virtual simulations can fully capture.
  • Future Work: The paper implies that future work will focus on incorporating soft tissue simulation and further enhancing the realism and accuracy of the surgical operations.

Summary updated

Abstract

<p style='text-indent:20px;'>Due to the complexity of the maxillofacial surgery, the novice should be sufficiently trained before one is qualified to carry on the surgery. To reduce the training costs and improve the training efficiency, a virtual mandible surgical system with haptic feedback is proposed. This surgical simulation system offers users the haptic feedback while simulating maxillofacial surgery. An integrated model is introduced to optimize the system simulation process, which includes force output to a six-degree-of-freedom haptic device. Based on the anatomy structure of the bone tissue, a two-layer mechanism model is designed to balance the requirement of real-time response and the force feedback accuracy. Collision detection, force rendering, and grinding function are studied to simulate some essential operations: open reduction, osteotomy, and palate fixation. The proposed simulation platform can assist in the training and planning of these oral and maxillofacial surgeries. The fast response feature enables surgeons to design a patient-specific guide plate in real-time. Ten stomatology surgeons evaluated this surgical simulation system from the following four indexes: the level of immersion, user-friendliness, stability, and the effect of surgical training. The evaluation score is eight out of ten.</p>
BibTeX
@article{zhang2020maxillofacial,
  author = {Jing Zhang and Jiahui Qian and Han Zhang and Ling He and Bin Li and Jing Qin and Hong‐Ning Dai and Wei Tang and Weidong Tian},
  title = {{Maxillofacial surgical simulation system with haptic feedback}},
  journal = {Journal of Industrial and Management Optimization},
  year = {2020},
  doi = {10.3934/jimo.2020137},
}
Figure