Abstract
Autonomous suturing has been a long-sought-after goal for surgical robotics. Outside of staged environments, accurate localization of suture needles is a critical foundation for automating various suture needle manipulation tasks in the real world. When localizing a needle held by a gripper, previous work usually tracks them separately without considering their relationship. Because of the significant errors that can arise in the stereo-triangulation of objects and instruments, their reconstructions may often not be consistent. This can lead to unrealistic tool-needle grasp reconstructions that are infeasible. Instead, an obvious strategy to improve localization would be to leverage constraints that arise from contact, thereby constraining reconstructions of objects and instruments into a jointly feasible space. In this work, we consider feasible grasping constraints when tracking the 6D pose of an in-hand suture needle. We propose a reparameterization trick to define a new state space for describing a needle pose, where grasp constraints can be easily defined and satisfied. Our proposed state space and feasible grasping constraints are then incorporated into Bayesian filters for real-time needle localization. In the experiments, we show that our constrained methods outperform previous unconstrained/constrained tracking approaches and demonstrate the importance of incorporating feasible grasping constraints into automating suture needle manipulation tasks.
Introduction
In minimally invasive robotic surgery, automating tasks like suturing requires precise knowledge of the needle's 6D pose (position and orientation). Traditionally, computer vision systems track the robotic tool and the needle as independent entities. However, due to sensor noise and the "thin-object" nature of needles, these independent tracks often produce inconsistent results.
As shown in the side-view reconstructions, unconstrained systems often depict the needle hovering centimeters away from the gripper or clipping through it. For a robot performing delicate maneuvers, these physically impossible poses lead to catastrophic failures. While complex physics simulators can theoretically resolve these issues, they are typically too computationally intensive for real-time surgical applications.
Methodology
The core innovation of this work is a Reparameterization Trick. Instead of tracking the needle using standard Cartesian coordinates ($x, y, z$ and Euler angles), the authors define a new 4D state space $(\alpha, w, u, v)$:
- $\alpha$: Represents the specific point along the needle's curve where it is being grasped.
- $w, u, v$: Describe the relative position and orientation of the gripper to that grasp point using a modified spherical coordinate system.
By using this specific mapping, the "feasible" space—where the needle is in contact with the gripper without collision—becomes a simple convex hyperrectangle.
This convexity is a critical mathematical property. It allows Bayesian filters (such as Histogram Filters or Particle Filters) to perform weighted averages of multiple pose hypotheses while guaranteeing that the final result remains a physically valid pose. This approach is "constrained by design," making it approximately 24 times faster (0.15s vs 3.6s per frame) than traditional rejection-sampling methods that simply discard invalid guesses.
Experimental Evaluation
The researchers evaluated their approach using two primary setups:
- Simulation (CoppeliaSim): A virtual environment featuring a surgical manipulator and a 5.4mm radius suture needle, allowing for testing against ground truth data under controlled noise levels.
- Real-World Ex-Vivo Data: Using the da Vinci Research Kit (dVRK), the team captured video of a Large Needle Driver (LND) manipulating needles of 7mm and 11.5mm radii. Needle features were detected using a Deep Neural Network (DeepLabCut).
Results and Discussion
The proposed constrained Particle Filter (cPFrp) demonstrated superior accuracy and robustness. In simulations, it maintained low positional and orientational error even as environmental noise increased, whereas unconstrained methods saw error spikes.
In a simulated "regrasping" task, the constrained method achieved high success rates under high-noise conditions, while unconstrained baselines failed nearly 100% of the time. Qualitatively, the constrained method produces 3D reconstructions that are visually consistent with physical reality, avoiding the "floating needle" artifacts common in independent tracking systems.
Conclusion
By transforming non-convex physical constraints into a convex mathematical state space, this research enables real-time, physically consistent 6D tracking for surgical needles. While currently assuming a stable grasp where the tool's y-axis intersects the needle, future work will aim to expand this framework to multi-needle scenarios and interactions with deformable soft tissues. This advancement represents a significant step toward reliable autonomous surgical manipulation.
Summary updated