Intra-operative Update of Boundary Conditions for Patient-Specific Surgical Simulation

Eleonora Tagliabue, Marco Piccinelli, Diego Dall'Alba, Juan Verde, Micha Pfeiffer, Riccardo Marin, Stefanie Speidel, Paolo Fiorini, Stéphane Cotin

In the field of surgical robotics and computer-assisted intervention, high-fidelity simulations are essential for navigation and autonomous task execution. However, making these simulations match the dynamic and often unpredictable environment of the operating room remains a significant challenge. This paper, presented at MICCAI 2021, introduces a real-time pipeline designed to update the "boundary conditions" (BCs)—such as adhesions and tissue attachments—of patient-specific biomechanical simulations using intra-operative vision data.

The Challenge: Dynamic Boundary Conditions

Patient-specific Biomechanical Models (PBMs) utilize pre-operative scans (CT or MRI) to provide geometric data. However, these scans rarely capture the specific points where organs are attached to one another via connective tissue or adhesions. Furthermore, these boundary conditions change during surgery as tissues are dissected. If a simulation does not account for these anchors, its predictions regarding tissue deformation and reaction to force will be fundamentally inaccurate.

Proposed Methodology

The researchers developed a two-process framework that integrates a real-time physics simulation (using the SOFA framework) with a continuous parameter update pipeline. The update process consists of five distinct stages:

  1. Pre-processing: Raw 3D point clouds from an RGB-D camera are segmented and aligned with the pre-operative model.
  2. Displacement Estimation: Utilizing the ZoomOut method, the system calculates Intra-Operative Displacement (IOD), mapping deformed surface points back to their original positions.
  3. Voxelization: The 3D geometry and displacement fields are converted into a $64 \times 64 \times 64$ grid to facilitate deep learning processing.
  4. BANet (Boundary Attachment Network): A U-Net-style Deep Neural Network (DNN) predicts which voxels represent attachment points based on observed surface deformations.
  5. PBM Update: The predicted BCs are mapped back to the simulation, creating an "Intra-Operative Model" (IOM) that reflects the current surgical state.
The five-step pipeline for updating the Patient-Specific Biomechanical Model (PBM) using intra-operative vision data.

Experimental Validation

The system was evaluated using two primary datasets:

  • Synthetic Dataset: 600 samples of adipose tissue manipulation were generated to provide ground truth knowledge for training BANet.
  • Ex Vivo Validation: The system was tested on a real human specimen involving the kidney, pararenal fat, and Gerota’s fascia. This test provided a realistic assessment of the system's performance against sensor noise, occlusions, and complex tissue behaviors.
Ex vivo experimental setup showing markers on human tissue used for validation.

Key Results and Insights

The central innovation of this work is the realization that deformation can serve as a proxy for connection. By observing how tissue stretches when manipulated, the neural network can "reverse-engineer" the location of hidden anchors.

  • Accuracy: In ex vivo human tissue tests, the average Root Mean Square Error (RMSE) remained below 5 mm, meeting the threshold for clinical acceptability in minimally invasive surgery.
  • Performance Gain: Simulations with updated BCs were significantly more accurate than unconstrained models, where errors reached as high as 31 mm.
  • Latency: The total pipeline latency was approximately 1.44 seconds. While displacement estimation accounts for the majority of this time (1.39s), the BANet inference is nearly instantaneous at 13 milliseconds.

Limitations and Future Directions

While the system demonstrates robust performance, several areas for improvement remain. The pipeline currently struggles with significant occlusions from surgical tools and high levels of surface noise during extreme deformations. Future work may involve incorporating temporal dynamics—treating the surgery as a sequence rather than static snapshots—and optimizing the system to handle complex topological changes such as tissue tearing or suturing.

Summary updated

Abstract

Patient-specific Biomechanical Models (PBMs) can enhance computer assisted surgical procedures with critical information. Although pre-operative data allow to parametrize such PBMs based on each patient’s properties, they are not able to fully characterize them. In particular, simulation boundary conditions cannot be determined from pre-operative modalities, but their correct definition is essential to improve the PBM predictive capability. In this work, we introduce a pipeline that provides an up-to-date estimate of boundary conditions, starting from the pre-operative model of patient anatomy and the displacement undergone by points visible from an intra-operative vision sensor. The presented pipeline is experimentally validated in realistic conditions on an ex vivo pararenal fat tissue manipulation. We demonstrate its capability to update a PBM reaching clinically acceptable performances, both in terms of accuracy and intra-operative time constraints.
BibTeX
@inproceedings{tagliabue2021intra,
  author = {Eleonora Tagliabue and Marco Piccinelli and Diego Dall'Alba and Juan Verde and Micha Pfeiffer and Riccardo Marin and Stefanie Speidel and Paolo Fiorini and Stéphane Cotin},
  title = {{Intra-operative Update of Boundary Conditions for Patient-Specific Surgical Simulation}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  doi = {10.1007/978-3-030-87202-1\_36},
}
Figure