Abstract
Digital twins are virtual interactive models of the real world, exhibiting identical behavior and properties. In surgical applications, computational analysis from digital twins can be used, for example, to enhance situational awareness. We present a digital twin framework for skull-base surgeries, named Twin-S, which can be integrated within various image-guided interventions seamlessly. Twin-S combines high-precision optical tracking and real-time simulation. We rely on rigorous calibration routines to ensure that the digital twin representation precisely mimics all real-world processes. Twin-S models and tracks the critical components of skull-base surgery, including the surgical tool, patient anatomy, and surgical camera. Significantly, Twin-S updates and reflects real-world drilling of the anatomical model in frame rate. We extensively evaluate the accuracy of Twin-S, which achieves an average 1.39 mm error during the drilling process. We further illustrate how segmentation masks derived from the continuously updated digital twin can augment the surgical microscope view in a mixed reality setting.
Introduction
Skull base surgeries, such as mastoidectomies, are high-stakes procedures requiring navigation through complex cortical bone to reach critical structures like nerves and major blood vessels. These structures are often invisible to the naked eye, buried within the bone. While Augmented Reality (AR) and Virtual Reality (VR) have been introduced to assist surgeons, existing systems often rely on static pre-operative scans that do not reflect intra-operative tissue removal.
Twin-S addresses this gap by creating a "live" digital counterpart that evolves alongside the physical surgery. By closing the loop between real-world measurements and virtual simulations, Twin-S provides a dynamic platform for surgical guidance, training, and data generation.
Methodology
The Twin-S framework establishes a continuous synchronization loop between the Physical Twin (the actual surgical setup) and the Digital Twin (the simulation environment).
System Architecture
The physical setup includes an optical tracker monitoring the 3D poses of the surgical drill, a patient phantom, and a stereo microscope. These poses are streamed into the Asynchronous Multi-Body Framework (AMBF), which hosts the digital twin. The virtual environment utilizes a pre-operative CT scan of the phantom to create a voxel-based occupancy map.
Real-Time Interaction and Calibration
As drilling occurs in the physical world, the system detects collisions between the virtual drill tip and the virtual bone model. Corresponding voxels are removed in real-time, updating the anatomical model at approximately 28 FPS. To maintain high fidelity, the system employs rigorous calibration routines, including pivot calibration for the drill, hand-eye calibration for the camera, and Iterative Closest Point (ICP) registration to align the virtual CT model with the physical phantom.
Results and Evaluation
The system was validated using a specialized surgical phantom and high-resolution CT scans. A surgical expert performed a simulated mastoidectomy, and the resulting "post-op" state was compared against the digital twin's model.
- Drilling Accuracy: Twin-S achieved an average error of 1.39 mm in drilled regions, comparable to current clinical optical navigation systems.
- Computational Performance: The total pipeline latency is roughly 35.7 ms, enabling a real-time update rate of 28 FPS.
- Segmentation Fidelity: The digital twin generated virtual segmentation masks with a Dice score of 0.956 for anatomy and 0.725 for the drill, demonstrating its potential for automated data labeling.
Conclusion
Twin-S provides a robust framework for real-time digital twinning in skull-base surgery. By updating the virtual model based on physical measurements, it enables dynamic feedback such as color-coded "safety buffer" visualizations. Future work will focus on integrating vision-based tracking to mitigate line-of-sight issues inherent in optical systems and leveraging the framework as a data engine for training surgical AI models.
Summary updated