Markerless Inside-Out Tracking for Interventional Applications

Benjamin Busam, Patrick Ruhkamp, Salvatore Virga, Beatrice Lentes, Nassir Navab, Christoph HennerspergerarXiv 2018 · OpenReview

Abstract

Traditional medical instrument tracking relies on external cameras, which are prone to line-of-sight occlusions and setup complexities. This paper introduces a markerless "inside-out" tracking system using visual Simultaneous Localization and Mapping (SLAM). By mounting a miniature RGB-D camera directly on an ultrasound probe, the system maps the operating room environment to determine its own pose. Experimental results demonstrate that this approach significantly improves rotational accuracy compared to commercial optical tracking systems, offering a robust alternative for cluttered clinical environments.

Introduction

Precise 3D tracking of medical instruments is a cornerstone of computer-assisted surgery and robotic guidance. Currently, the gold standard is "outside-in" optical tracking systems (OTS), which utilize external infrared cameras to monitor reflective markers on tools. However, these systems are limited by line-of-sight (LoS) requirements; in crowded operating rooms, personnel or equipment frequently obstruct the camera's view, leading to tracking failures. Furthermore, the need for specialized markers adds complexity to the clinical workflow.

Methodology

To address these limitations, the researchers developed an "inside-out" tracking prototype. The system utilizes an Intel RealSense D435 camera—a compact sensor providing RGB, stereo infrared, and depth data—integrated onto a transrectal ultrasound (TRUS) probe.

Hardware architecture and coordinate frames for the inside-out tracking system.

The core software utilizes Visual SLAM to localize the probe. Unlike traditional methods, SLAM identifies natural environmental features (e.g., equipment edges, table corners) to build a 3D map and calculate the camera's position within it. Two algorithmic pipelines were evaluated: ORB-SLAM2 (feature-based) and Stereo-DSO (direct sparse odometry).

Experimental Setup

Validation was conducted in a realistic prostate biopsy environment. A KUKA iiwa robotic arm provided ground truth measurements (±0.1 mm accuracy), while a Polaris Vicra OTS served as a commercial baseline. The environment was intentionally filled with clinical equipment to provide a feature-rich environment for the SLAM algorithms.

The clinical experimental setup featuring the ultrasound phantom and examination bed used for validation.

Key Insights

A primary advantage of inside-out tracking is the Rotation Leverage Effect. Because the camera observes distant environmental features, minor rotations of the probe result in significant shifts in the visual field, enhancing rotational sensitivity compared to external systems observing small, proximal markers. Additionally, clinical "clutter" acts as a benefit for SLAM, providing a rich set of high-contrast visual features for robust tracking.

Results

The feature-based ORB-SLAM2 pipeline emerged as the most effective method in the study's custom dataset of over 8,000 poses.

  • Rotational Accuracy: ORB-SLAM2 achieved an angular error of 1.99°, significantly outperforming the commercial OTS (8.43°).
  • Translational Accuracy: The commercial OTS maintained a slight advantage (1.90 mm error) over ORB-SLAM2 (2.65 mm), though the latter remains functional for many interventional tasks.
  • Qualitative Performance: 3D ultrasound reconstructions using SLAM tracking exhibited smoother surfaces and clearer boundaries than those using OTS, directly attributable to superior rotational precision.

Discussion and Future Work

While the prototype demonstrates superior rotational tracking and eliminates LoS issues, challenges remain. Future work will focus on reducing translational lag through sensor fusion with Inertial Measurement Units (IMUs) and optimizing computational efficiency for real-time clinical use. Furthermore, "collaborative SLAM" could allow multiple instruments to share a single environmental map, facilitating complex multi-tool coordination without external infrastructure.

Summary updated

Abstract

Tracking of rotation and translation of medical instruments plays a substantial role in many modern interventions. Traditional external optical tracking systems are often subject to line-of-sight issues, in particular when the region of interest is difficult to access or the procedure allows only for limited rigid body markers. The introduction of inside-out tracking systems aims to overcome these issues. We propose a marker-less tracking system based on visual SLAM to enable tracking of instruments in an interventional scenario. To achieve this goal, we mount a miniature multi-modal (monocular, stereo, active depth) vision system on the object of interest and relocalize its pose within an adaptive map of the operating room. We compare state-of-the-art algorithmic pipelines and apply the idea to transrectal 3D Ultrasound (TRUS) compounding of the prostate. Obtained volumes are compared to reconstruction using a commercial optical tracking system as well as a robotic manipulator. Feature-based binocular SLAM is identified as the most promising method and is tested extensively in challenging clinical environment under severe occlusion and for the use case of prostate US biopsies.
BibTeX
@misc{busam2018markerless,
  author = {Benjamin Busam and Patrick Ruhkamp and Salvatore Virga and Beatrice Lentes and Nassir Navab and Christoph Hennersperger},
  title = {{Markerless Inside-Out Tracking for Interventional Applications}},
  year = {2018},
  eprint = {1804.01708},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1804.01708},
}
Figure