Abstract
Suture needle localization is necessary for autonomous suturing. Previous approaches in autonomous suturing often relied on fiducial markers rather than markerless detection schemes for localizing a suture needle due to the inconsistency of markerless detections. However, fiducial markers are not practical for real-world applications and can often be occluded from environmental factors in surgery (e.g., blood). Therefore in this work, we present a robust tracking approach for estimating the 6D pose of a suture needle when using inconsistent detections. We define observation models based on suture needles' geometry that captures the uncertainty of the detections and fuse them temporally in a probabilistic fashion. In our experiments, we compare different permutations of the observation models in the suture needle localization task to show their effectiveness. Our proposed method outperforms previous approaches in localizing a suture needle. We also demonstrate the proposed tracking method in an autonomous suture needle regrasping task and ex vivo environments.
Introduction
Autonomous suturing represents a significant milestone in minimally invasive robotic surgery. A fundamental requirement for this task is the precise, real-time localization of the suture needle. Traditionally, researchers have relied on fiducial markers—small, high-contrast dots—to track needles. However, these markers are clinically impractical as they can be obscured by blood, tissue, or surgical debris, or simply wear off during the procedure.
Markerless detection offers a more viable clinical alternative but is plagued by "jitter" and inconsistency. Visual detections from an endoscope are often noisy due to glare or partial occlusions. This research addresses these challenges by developing a robust framework to transform inconsistent 2D image detections into a stable 3D (6D pose) trajectory suitable for autonomous robotic manipulation.
Methodology
The proposed system utilizes a Particle Filter (PF) framework to fuse temporal information and robotic kinematics. Rather than relying on a single detection per frame, the filter maintains a distribution of thousands of potential needle poses ("particles"), updating their likelihood based on new visual evidence and the known movements of the robotic arm.
Points Matching to Ellipse (PM-E) Model
The core innovation is the Points Matching to Ellipse (PM-E) observation model. Since a circular needle projects onto a 2D camera sensor as an ellipse, the model evaluates how well detected pixels align with the projected ellipse of a candidate 3D pose.
Key features of this methodology include:
- Uncertainty Modeling: The system analytically derives the uncertainty of the ellipse fit based on pixel-level noise, allowing the filter to dynamically adjust its trust in detections.
- Anchor Points: To resolve the symmetry of an ellipse (which makes it difficult to distinguish the tip from the tail), the model tracks specific "anchor" points, such as the tail where the thread is attached.
Experimental Evaluation
The approach was validated across three distinct environments:
- CoppeliaSim Simulation: Used for benchmarking against state-of-the-art methods like Feature Point Similarity (FPS) under controlled noise conditions.
- Real-World dVRK: Implemented on the da Vinci Research Kit using stereo endoscopic cameras. Detections were generated via DeepLabCut, trained on a small set of manually labeled surgical images.
- Ex Vivo Testing: Conducted on real tissue samples to ensure robustness against the complex, non-uniform backgrounds of actual surgical sites.
Key Results and Insights
The proposed "2 points + EM" (Ellipse Matching) model demonstrated significant improvements over existing techniques:
- Accuracy: In high-noise simulations, the system achieved a tracking error of only 0.87 mm in position and 0.12 degrees in orientation, vastly outperforming traditional methods that often exceeded 10mm of error under similar conditions.
- Robustness: The system maintained an 80% success rate in autonomous needle regrasping tasks, even when visual detections were highly inconsistent.
- Efficiency: The observation model processes at approximately 30 FPS (0.03 seconds per frame), meeting the requirements for real-time robotic control.
Conclusion and Future Work
By treating needle pose as a probability distribution rather than a static point, this system can recover from temporary tracking failures and occlusions. While the model is highly effective, future work will focus on resolving axial rotation ambiguity and incorporating physical constraints—such as the mathematical relationship between the needle and the robotic gripper—to further enhance tracking stability during complex maneuvers.
Summary updated