Deformable Gaussian Splatting for Efficient and High-Fidelity Reconstruction of Surgical Scenes

Jiwei Shan, Zeyu Cai, Cheng-Tai Hsieh, Lijun Han, Shing Shin Cheng, Hesheng Wang

Abstract

Real-time 3D reconstruction of surgical scenes is critical for computer-assisted interventions but faces significant challenges due to soft tissue deformation and topological changes such as cutting or shearing. This paper presents EH-SurGS, a framework that leverages 3D Gaussian Splatting (3DGS) enhanced with a temporal "life cycle" modeling approach and an adaptive motion hierarchy. EH-SurGS effectively handles irreversible tissue changes and minimizes computational waste in static regions. Experimental results on EndoNeRF, StereoMIS, and Hamlyn datasets demonstrate that EH-SurGS achieves state-of-the-art visual fidelity (up to 39.91 PSNR) while maintaining rendering speeds of approximately 400 FPS, significantly outperforming existing NeRF and 3DGS-based methods.

Introduction

Reconstructing 3D surgical scenes from endoscopic video is complicated by the dynamic nature of soft tissues, which deform due to physiological motion (breathing) and tool interactions. Traditional Neural Radiance Fields (NeRF) often fail to meet the real-time requirements of clinical settings due to slow rendering speeds. While 3D Gaussian Splatting (3DGS) offers faster performance, it struggles with two primary issues: irreversible topological changes, such as tissue shearing or cutting, and the inefficiency of applying global deformation models to scenes containing large static backgrounds.

Methodology

EH-SurGS introduces two core innovations to address these limitations within the 3DGS pipeline.

Deformation Modeling with Life Cycle

Standard deformation models assume a continuous mapping from a canonical state to a deformed state, which breaks down during surgical procedures involving tissue removal or incision. EH-SurGS introduces a life cycle for each 3D Gaussian by incorporating learnable temporal opacity. Using Gaussian basis functions, the model controls the "activation" and "deactivation" of Gaussians over time. This allows the system to represent newly exposed surfaces or removed tissue without the "ghosting" artifacts typical of purely spatial warping methods.

Adaptive Motion Hierarchy Strategy

To optimize rendering performance, EH-SurGS employs an adaptive hierarchy to distinguish between static and deformable regions. The system monitors average deformation and rendering loss to identify dynamic areas. If a region contains mixed signals, it is subdivided for finer granularity. Gaussians in identified static regions bypass the deformation field entirely, drastically reducing GPU overhead.

EH-SurGS Pipeline: The framework integrates life cycle deformation modeling with an adaptive motion hierarchy to optimize both fidelity and speed.

Experimental Results

The framework was evaluated on three major endoscopic datasets: EndoNeRF (robotic prostatectomy), StereoMIS (da Vinci Xi clips), and the Hamlyn Dataset (soft tissue cutting).

EH-SurGS consistently outperformed state-of-the-art (SOTA) methods, including Deform3DGS and Endo-4DGS. Notably, it achieved a PSNR of 39.91 on the EndoNeRF-Cutting sequence. In terms of efficiency, the model reached rendering speeds between 379 and 400 FPS on an NVIDIA RTX 4090, representing a massive improvement over NeRF-based models that often operate below 2 FPS. Training times are also competitive, typically requiring only 100-130 seconds.

Performance comparison: EH-SurGS occupies the top-right quadrant, demonstrating the highest PSNR and FPS among recent MICCAI publications.

Conclusion

EH-SurGS represents a significant step forward in surgical scene reconstruction by combining temporal life cycle modeling with selective computation. By focusing GPU resources only on dynamic pixels, the authors achieved a substantial jump in FPS without sacrificing visual quality. While currently optimized for static endoscope settings, future work aims to integrate SLAM capabilities to handle dynamic camera motion, further expanding its utility in complex surgical environments.

Summary updated

BibTeX
@inproceedings{shan2025deformable,
  author = {Jiwei Shan and Zeyu Cai and Cheng-Tai Hsieh and Lijun Han and Shing Shin Cheng and Hesheng Wang},
  title = {{Deformable Gaussian Splatting for Efficient and High-Fidelity Reconstruction of Surgical Scenes}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2025},
  doi = {10.1109/icra55743.2025.11128551},
}
Figure