Abstract
Real-time 3D reconstruction of surgical scenes is critical for computer-assisted interventions but faces significant challenges due to soft tissue deformation and topological changes such as cutting or shearing. This paper presents EH-SurGS, a framework that leverages 3D Gaussian Splatting (3DGS) enhanced with a temporal "life cycle" modeling approach and an adaptive motion hierarchy. EH-SurGS effectively handles irreversible tissue changes and minimizes computational waste in static regions. Experimental results on EndoNeRF, StereoMIS, and Hamlyn datasets demonstrate that EH-SurGS achieves state-of-the-art visual fidelity (up to 39.91 PSNR) while maintaining rendering speeds of approximately 400 FPS, significantly outperforming existing NeRF and 3DGS-based methods.
Introduction
Reconstructing 3D surgical scenes from endoscopic video is complicated by the dynamic nature of soft tissues, which deform due to physiological motion (breathing) and tool interactions. Traditional Neural Radiance Fields (NeRF) often fail to meet the real-time requirements of clinical settings due to slow rendering speeds. While 3D Gaussian Splatting (3DGS) offers faster performance, it struggles with two primary issues: irreversible topological changes, such as tissue shearing or cutting, and the inefficiency of applying global deformation models to scenes containing large static backgrounds.
Methodology
EH-SurGS introduces two core innovations to address these limitations within the 3DGS pipeline.
Deformation Modeling with Life Cycle
Standard deformation models assume a continuous mapping from a canonical state to a deformed state, which breaks down during surgical procedures involving tissue removal or incision. EH-SurGS introduces a life cycle for each 3D Gaussian by incorporating learnable temporal opacity. Using Gaussian basis functions, the model controls the "activation" and "deactivation" of Gaussians over time. This allows the system to represent newly exposed surfaces or removed tissue without the "ghosting" artifacts typical of purely spatial warping methods.
Adaptive Motion Hierarchy Strategy
To optimize rendering performance, EH-SurGS employs an adaptive hierarchy to distinguish between static and deformable regions. The system monitors average deformation and rendering loss to identify dynamic areas. If a region contains mixed signals, it is subdivided for finer granularity. Gaussians in identified static regions bypass the deformation field entirely, drastically reducing GPU overhead.
Experimental Results
The framework was evaluated on three major endoscopic datasets: EndoNeRF (robotic prostatectomy), StereoMIS (da Vinci Xi clips), and the Hamlyn Dataset (soft tissue cutting).
EH-SurGS consistently outperformed state-of-the-art (SOTA) methods, including Deform3DGS and Endo-4DGS. Notably, it achieved a PSNR of 39.91 on the EndoNeRF-Cutting sequence. In terms of efficiency, the model reached rendering speeds between 379 and 400 FPS on an NVIDIA RTX 4090, representing a massive improvement over NeRF-based models that often operate below 2 FPS. Training times are also competitive, typically requiring only 100-130 seconds.
Conclusion
EH-SurGS represents a significant step forward in surgical scene reconstruction by combining temporal life cycle modeling with selective computation. By focusing GPU resources only on dynamic pixels, the authors achieved a substantial jump in FPS without sacrificing visual quality. While currently optimized for static endoscope settings, future work aims to integrate SLAM capabilities to handle dynamic camera motion, further expanding its utility in complex surgical environments.
Summary updated