Markerless Suture Needle 6D Pose Tracking with Robust Uncertainty Estimation for Autonomous Minimally Invasive Robotic Surgery

Zih-Yun Chiu, Albert Z. Liao, Florian Richter, Bjorn Johnson, Michael C. Yip

TL;DR

Researchers at UC San Diego have developed a robust, markerless system for tracking the 6D pose of suture needles during robotic surgery. By combining a Particle Filter with a novel "Points Matching to Ellipse" observation model, the system can accurately estimate a needle's position and orientation even when visual detections are noisy or partially occluded, paving the way for fully autonomous surgical suturing.

What Problem Does This Paper Address?

Autonomous suturing is a "holy grail" in minimally invasive robotic surgery, but it requires the robot to know exactly where the needle is at all times. Most existing systems rely on fiducial markers (like tiny colored dots) placed on the needle. However, markers are impractical for real clinical use: they can be rubbed off, obscured by blood, or simply too small to see clearly through an endoscope.

Markerless detection is the alternative, but it is notoriously "jittery" and inconsistent. A single frame might only capture a few pixels of the needle, or the detection might be slightly off due to glare. This paper addresses the challenge of taking these inconsistent, noisy 2D image detections and turning them into a stable, 3D (6D pose) trajectory that a robot can use to perform complex tasks like regrasping or stitching.

Methods

The authors use a Particle Filter (PF) framework to fuse temporal information. Instead of relying on a single "best guess" from one image, the filter maintains thousands of "particles" (hypothetical needle poses) and updates their likelihood based on new evidence.

The core innovations include:

  • Motion Model: The system doesn't just guess where the needle went; it incorporates the kinematics of the robotic needle driver. If the robot arm moves 5mm to the left, the filter expects the needle to move accordingly.
  • Points Matching to Ellipse (PM-E) Observation Model: This is the paper's primary contribution. Because a circular suture needle projects onto a 2D camera as an ellipse, the authors developed a way to measure how well detected pixels "fit" the projected ellipse of a candidate 3D pose.
  • Uncertainty Modeling: Unlike previous methods that treat detections as ground truth, this model analytically derives the uncertainty of the ellipse fit based on pixel-level noise. This allows the filter to "trust" detections less when they are messy.

Datasets

The researchers evaluated their approach using two main setups:

  1. CoppeliaSim Simulation: A virtual environment where they could control noise levels ($\sigma$) to benchmark against previous state-of-the-art methods like Feature Point Similarity (FPS) and Normalized Cross-Correlation Similarity (NCCS).
  2. Real-World dVRK: They used the da Vinci Research Kit (dVRK) with a stereo endoscopic camera. Detections were generated using DeepLabCut, a popular deep-learning tool for pose estimation, trained on a small set of manually labeled surgical images.
  3. Ex Vivo Environments: Testing was performed on real tissue samples to demonstrate robustness against the complex backgrounds found in actual surgery.

Key Insights and Ideas

  • Solving the Correspondence Problem: Standard tracking often requires knowing exactly which part of the needle a pixel represents (e.g., "this pixel is exactly 3mm from the tip"). The PM-E model bypasses this by simply asking, "Does this pixel lie anywhere on the needle's body?" This makes it much more robust to partial occlusions.
  • The "Anchor" Requirement: An ellipse alone is symmetric, meaning the computer can't tell the tip from the tail just by looking at the curve. The authors found that tracking just one or two specific points (like the tail where the thread attaches) acts as an "anchor" to resolve this ambiguity.
  • Probabilistic Fusion beats Raw Optimization: By treating the needle pose as a probability distribution rather than a single point to be optimized, the system can recover from temporary tracking failures or sudden movements that would crash traditional "best-fit" algorithms.

Key Results

  • Superior Accuracy: In simulations with high noise, the proposed "2 points + EM" (Ellipse Matching) model achieved a tracking error of just 0.87 mm in position and 0.12 degrees in orientation. This significantly outperformed previous methods, which often saw errors jump to over 10mm under the same noise conditions.
  • High Success Rates: When integrated into an autonomous "needle regrasping" task (where the robot must grab a needle, let go, and grab it again), the system maintained a success rate of over 80% even in highly noisy environments.
  • Real-Time Performance: The observation model is computationally efficient, processing at approximately 0.03 seconds per frame (roughly 30 FPS), making it suitable for live robotic control.

Limitations and Future Work

  • Axial Ambiguity: While the model is excellent at tracking the needle's curve, it still struggles slightly with rotation around the needle's own thin axis if no distinct features (like the thread attachment) are visible.
  • Physical Constraints: Currently, the filter doesn't "know" that the needle is physically stuck inside a gripper. The authors suggest that future work could use Constrained Bayesian Estimation to incorporate the fact that the needle's pose is mathematically tied to the gripper's pose when being held.
  • Complex Occlusions: While it handles partial occlusion well, total occlusion (e.g., the needle disappearing entirely behind a fold of tissue) remains a challenge for any vision-based system.

Summary updated

Abstract

Suture needle localization is necessary for autonomous suturing. Previous approaches in autonomous suturing often relied on fiducial markers rather than markerless detection schemes for localizing a suture needle due to the in-consistency of markerless detections. However, fiducial markers are not practical for real-world applications and can often be occluded from environmental factors in surgery (e.g., blood). Therefore in this work, we present a robust tracking approach for estimating the 6D pose of a suture needle when using inconsistent detections. We define observation models based on suture needles' geometry that captures the uncertainty of the detections and fuse them temporally in a probabilistic fashion. In our experiments, we compare different permutations of the observation models in the suture needle localization task to show their effectiveness. Our proposed method outperforms previous approaches in localizing a suture needle. We also demonstrate the proposed tracking method in an autonomous suture needle regrasping task and ex vivo environments**The code is available at https://github.com/ucsdarclab/suture-needle-tracking..
BibTeX
@inproceedings{chiu2022markerless,
  author = {Zih-Yun Chiu and Albert Z. Liao and Florian Richter and Bjorn Johnson and Michael C. Yip},
  title = {{Markerless Suture Needle 6D Pose Tracking with Robust Uncertainty Estimation for Autonomous Minimally Invasive Robotic Surgery}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=8WX_wMchUN},
}

Loading related content...

Figure