ColibriDoc: An Eye-in-Hand Autonomous Trocar Docking System

Shervin Dehghani, Michael Sommersperger, Junjie Yang, Benjamin Busam, Kai Huang, Peter Gehlbach, Iulian Iordachita, Nassir Navab, M. Ali Nasseri

Introduction

Vitreoretinal surgery requires extreme precision, often operating at a micron-level scale. To access the internal structures of the eye, surgeons must first insert tiny ports called trocars into the sclera. While robotic assistants have become invaluable for stabilizing a surgeon's hand during internal maneuvers, the initial "docking" phase—inserting the surgical instrument into the trocar—remains a significant challenge.

Currently, this process is performed manually using joysticks to navigate the robot. This task is not only time-consuming but also cognitively taxing, as the small workspace and delicate environment leave no room for error. Misalignment during docking can lead to unnecessary trauma to the eye. ColibriDoc addresses this by automating the entry phase, allowing surgeons to focus their attention on the core surgical procedure.

System Architecture

The ColibriDoc system utilizes an "Eye-in-Hand" configuration. In this setup, a monocular RGB camera is mounted directly onto the robot's end-effector, providing a close-up view of the surgical field from the perspective of the instrument tip.

The physical robotic setup showing (a) the camera, (b) the robotic arm, (c) the surgical needle, and (d) the phantom eye model.

Algorithmic Pipeline

The autonomous docking process is driven by a two-stage deep learning pipeline designed to handle the visual complexities of the surgical environment:

  1. Trocar Entry Point (TEP) Detection: A U-Net-style network with a ResNet34 backbone identifies the precise 2D coordinates of the trocar's opening within the camera feed.
  2. 4DoF Pose Estimation: Once the TEP is located, a second ResNet34-based network estimates the 3D orientation (pose) of the trocar. To manage the symmetrical and often texture-less nature of metallic trocars, the authors employed a specialized 6D rotation representation and a rotation-invariant loss function ($L_{rotation}$) that ignores irrelevant rotations around the trocar's own Z-axis.

Biomimetic Trajectory Planning

Inspired by the flight behavior of the Cuban Colibri (hummingbird), the system employs a specific three-stage approach to docking. Rather than moving directly toward the target, which can be difficult with a single camera due to depth uncertainty, the robot mimics the bird's alignment strategy:

  • Identify: The system locates the trocar and aligns the camera's orientation.
  • Approach: The robot adjusts its XY translation to center the tool directly over the trocar opening.
  • Dock: The robot moves along the Z-axis (the "beak" approach) to perform the final insertion.
The three-stage autonomous docking sequence: (1) Identify the target, (2) Approach and align, and (3) Dock the instrument.

Experimental Results

The system was evaluated using a combination of synthetic data and real-world phantom eye models. Key performance metrics include:

  • Precision: TEP detection achieved a mean error of just 0.74% (approximately 3 pixels on a 720p display).
  • Orientation: 80% of orientation estimates were within 10° of the ground truth.
  • Efficiency: In live trials, ColibriDoc successfully docked the instrument in 10 out of 11 attempts. The autonomous system completed the task in an average of 35.1 seconds, notably faster than trained biomedical engineers using manual joystick control, who averaged 40.8 seconds with significantly higher variance.

Future Outlook

While ColibriDoc demonstrates the feasibility of autonomous docking, several areas for refinement remain. Future work will focus on increasing the "searching" distance to allow docking from further away and adapting trajectory planning for non-linear "bent-tip" tools. Additionally, the integration of force-sensing and automated insertion depth detection (via surgical microscope feedback) will be critical for ensuring patient safety in clinical environments.

Summary updated

BibTeX
@inproceedings{dehghani2022colibridoc,
  author = {Shervin Dehghani and Michael Sommersperger and Junjie Yang and Benjamin Busam and Kai Huang and Peter Gehlbach and Iulian Iordachita and Nassir Navab and M. Ali Nasseri},
  title = {{ColibriDoc: An Eye-in-Hand Autonomous Trocar Docking System}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
  doi = {10.1109/icra46639.2022.9811364},
}
Figure