Abstract
In surgical oncology, screening colonoscopy is vital for diagnostic assistance and surgical navigation. However, accurate 3D surface reconstruction from monocular endoscopic videos remains challenging due to constrained camera movement, inadequate tissue texture, and scale ambiguity. We introduce a novel method for colon section reconstruction by leveraging Neural Implicit Surfaces (NeuS) supplemented by a single frame of depth map. By utilizing a cross-domain depth estimator and an adaptive regularization technique, our approach achieves stable, consistently scaled reconstructions. Validation on phantom imagery demonstrates exceptional accuracy in rendering colon sections, even capturing unseen surface portions, promising enhanced quality in cancer screening and treatment interventions.
Introduction
Colonoscopy is the gold standard for colorectal cancer screening, yet navigating the "pink tunnel" of the colon is notoriously difficult. Surgeons often lack a clear 3D understanding of the environment, which can lead to missed polyps or incomplete examinations. While Neural Radiance Fields (NeRF) and Neural Implicit Surfaces (NeuS) have revolutionized 3D reconstruction, they struggle in endoscopy due to the featureless nature of colon tissue, constrained camera paths, and the "confused scale" problem inherent in monocular vision.
Methodology
Our solution builds upon Neural Implicit Surfaces (NeuS), representing 3D surfaces as a Signed Distance Function (SDF). We introduce three key innovations to address the specific challenges of endoscopic imaging:
Cross-Domain Depth Guidance
To avoid the need for massive labeled medical datasets, we utilize a DPT Hybrid model pre-trained on natural scenes. These depth predictions are rescaled to the specific endoscopic scene using a sparse point cloud generated via Structure-from-Motion (COLMAP).
Single-Frame Anchoring
We demonstrate that even a single frame of depth information can act as a powerful prior to stabilize the entire reconstruction and resolve scale issues across the sequence.
Adaptive Eikonal Regularization
Standard SDF training uses a fixed Eikonal loss, which often leads to over-smoothed or "flat" reconstructions in low-texture areas. We introduced a ray-wise weight ($\lambda_r$) that adaptively relaxes this constraint based on the distance between the predicted depth and the current surface estimate. This allows the model to capture finer geometric details while maintaining mathematical validity.
Experimental Results
We evaluated our model using the C3VD (Colonoscopy 3D Video Dataset), which contains high-resolution video sequences from four distinct colon phantom regions: cecum, descending, sigmoid, and transcending.
Quantitative Performance
Our method was benchmarked against Gaussian NeRF and Vanilla NeRF. As shown in the table below, our approach consistently outperformed or matched the baselines in terms of Peak Signal-to-Noise Ratio (PSNR) across 11 different sub-sequences. Notably, in the "c1_a" sequence, we achieved a PSNR of 34.27, significantly higher than the 32.84 and 32.71 achieved by Gaussian and Vanilla NeRF, respectively.
Qualitative Performance
Visual inspections reveal that our method captures the complex folds of the colon wall with high fidelity. Because the model learns a continuous implicit field, it can effectively "fill in the gaps" and render portions of the colon surface that were not directly captured in the original video frames, a feat difficult for traditional MVS approaches.
Discussion and Conclusion
This work demonstrates that depth estimators trained on everyday objects can provide a useful geometric skeleton for medical tasks when properly rescaled. By making regularization adaptive, we overcome the geometric bias that typically plagues neural surfaces in low-texture environments.
Future work will focus on addressing the static scene assumption, as the real colon is a deformable organ. Additionally, we aim to reduce the dependency on COLMAP for initial scale alignment and validate the model on clinical data containing fluids and surgical tools. Despite these challenges, this breakthrough opens new avenues for achieving stable and consistently scaled reconstructions in clinical practice.
Summary updated