Abstract
Accurately simulating soft tissue deformation is crucial for surgical training, pre-operative planning, and real-time haptic feedback systems. While physics-based models such as the finite element method (FEM) provide high-fidelity results, they are often computationally expensive and require extensive preprocessing. We propose a graph neural network (GNN) architecture that predicts both tissue surface deformation and applied force from sparse point clouds. The model incorporates internal anatomical information through binary tissue profiles beneath each point and leverages E(n)-equivariant message passing to improve robustness. We collected experimental data that comprises a real silicone and bone-like phantom, and complemented it with synthetic simulations generated using FEM. Our model achieves a comparable performance to a baseline GNN on standard test cases and significantly outperforms it in rotated and cross-resolution scenarios, showing a strong generalization to unseen orientations and point densities. It also achieves a significant speed improvement, offering a solution for real-time applications. When fine-tuned on experimental data, the model maintains sub-millimeter deformation accuracy despite limited sample size and measurement noise. The results demonstrate that our approach offers an efficient, data-driven alternative to traditional simulations, capable of generalizing across anatomical configurations and supporting interactive surgical environments.
Introduction
Simulating how human tissue deforms under a surgeon’s touch is a high-stakes balancing act. Traditional Finite Element Method (FEM) simulations are the gold standard for accuracy but are often too computationally heavy for real-time applications like haptic feedback. Conversely, standard deep learning models are fast but frequently "blind" to internal anatomy or fail when the camera or organ orientation changes. This paper introduces a specialized Graph Neural Network (GNN) architecture designed to bridge this gap, offering high-fidelity, real-time tissue simulation that accounts for internal structures like bone.
Methodology
The core of the proposed approach is a Conditional Equivariant Graph Convolutional Layer (cEGCL). This architecture addresses three major hurdles in surgical simulation: heterogeneity, rotational variance, and resolution sensitivity.
Key innovations include:
- E(n)-Equivariance: Unlike standard networks, equivariant layers are inherently aware of rotational physics. If the input tissue rotates, the predicted deformation rotates perfectly with it, eliminating the need for massive data augmentation.
- Internal Tissue Profiles: For every point on the surface, the model encodes a 128-dimensional binary vector representing the vertical distribution of soft tissue versus bone. This allows the model to "feel" hard structures hidden beneath the surface.
- Dual-Branch Architecture: The network splits into two paths: one predicting the 3D displacement of surface points and the other predicting the scalar force applied by a surgical tool.
Experimental Setup and Data Collection
The researchers validated their model using a hybrid data approach. They generated approximately 12,000 synthetic simulations using FEM to model a soft silicone block with a complex, 3D-printed "bone" structure inside. This was complemented by experimental data from a physical phantom, using a robotic indenter equipped with a force sensor and motion capture system to record 1,315 real-world deformation states.
Key Results and Insights
- Real-Time Performance: The model performs a single simulation in just 0.015 seconds, nearly four times faster than baseline GNNs and orders of magnitude faster than traditional FEM.
- Rotational Robustness: While standard models saw force errors triple under rotation, the equivariant model remained stable, demonstrating superior generalization to unseen orientations.
- Zero-Shot Resolution Scaling: A model trained on sparse data (40 points) successfully predicted deformations on a dense grid (1024 points) without retraining, suggesting it learns underlying physics rather than simple point memorization.
- Accuracy: The system achieved sub-millimeter deformation accuracy (0.156 mm mean error) on synthetic data and maintained meaningful force prediction (0.7 N error) on noisy experimental data.
Conclusion
By leveraging equivariant graph neural networks and internal anatomical profiles, this approach provides a robust, data-driven alternative to traditional surgical simulations. The model's ability to generalize across orientations and point densities makes it a promising candidate for real-time haptic feedback and patient-specific surgical planning.
Summary updated