MBAPose: Mask and Bounding-Box Aware Pose Estimation of Surgical Instruments with Photorealistic Domain Randomization

Masakazu Yoshimura, Murilo Marques Marinho, Kanako Harada, Mamoru Mitsuishi

Abstract

Surgical robots are usually controlled using a priori models based on the robots' geometric parameters, which are calibrated before the surgical procedure. One of the challenges in using robots in real surgical settings is that those parameters can change over time, consequently deteriorating control accuracy. In this context, our group has been investigating online calibration strategies without added sensors. In one step toward that goal, we have developed an algorithm to estimate the pose of the instruments' shafts in endoscopic images. In this study, we build upon that earlier work and propose a new framework to more precisely estimate the pose of a rigid surgical instrument. Our strategy is based on a novel pose estimation model called MBAPose and the use of synthetic training data. Our experiments demonstrated an improvement of 21 % for translation error and 26 % for orientation error on synthetic test data with respect to our previous work. Results with real test data provide a baseline for further research.

Introduction

Surgical robotics relies heavily on the precise calibration of geometric parameters. However, these parameters often drift during procedures due to mechanical stress, temperature changes, or interactions with tissue, necessitating online re-calibration to maintain control accuracy. While computer vision offers a potential solution for "on-the-fly" calibration, training robust models is hindered by the "domain gap" between synthetic training data and the complex, reflective reality of surgical environments. This paper introduces MBAPose, a framework designed to bridge this gap through photorealistic rendering and advanced domain randomization.

Methods

The MBAPose architecture utilizes a single-stage pose estimation approach with a double-encoder backbone (ResNet34 and ResNet18). A key innovation is Mask Guidance: during training, the model is forced to predict the instrument's silhouette (segmentation mask). This awareness of the object's contour significantly enhances its ability to calculate 3D orientation parameters like pitch and yaw.

To address the domain gap, the researchers employed Physically-Based Rendering (PBR) within a virtual environment. Instruments were placed inside a virtual cube textured with real surgical background images, allowing metallic tools to realistically reflect their surroundings. Furthermore, a comprehensive Domain Randomization (DR) strategy was implemented, covering eleven distinct dimensions (dr1-dr11), including lighting position, intensity, and tool surface imperfections like metallic scratches simulated via normal mapping.

Overview of domain randomization techniques used to bridge the reality gap.

Datasets and Markerless Calibration

Training was conducted on a synthetic dataset of 90,000 photorealistic images generated in Blender. For real-world evaluation, the authors used 5,301 images captured using a "BionicBrain" (an anatomically realistic head model). To establish ground truth for these real images, a custom markerless calibration pipeline was developed. This process involves manually annotating tool contours and then iteratively updating the 3D CAD model's projection to maximize correspondence with the manual mask and edges.

The markerless calibration pipeline for real-world data labeling.

Key Insights and Results

Experimental results showed that MBAPose significantly outperforms previous methods, achieving a 21% improvement in translation error and a 26% improvement in orientation error on synthetic data.

  • Synthetic Accuracy: The model achieved a translation error of 0.88 mm and an orientation error of 2.66°.
  • Real-World Performance: With minimal fine-tuning (using only 50 real images), the model maintained a translation error of 2.18 mm and an orientation error of 5.11°.
  • Efficiency: Running at approximately 79.4 FPS on a Titan RTX GPU, the system is well-suited for real-time surgical use.

A critical insight from the study was the "glossiness trap": researchers found that excessive randomization of tool glossiness can actually hurt performance, as specific specular reflections are vital cues the AI uses for shape understanding. Additionally, adding "scratches" via normal mapping proved more effective than simple color randomization for preparing the model for real-world visual noise.

Conclusion and Future Work

MBAPose demonstrates that combining photorealistic domain randomization with mask-aware architectures can effectively estimate surgical tool poses from standard endoscopic images. While the current system operates frame-by-frame, future work will focus on incorporating temporal information from video sequences to improve robustness against occlusions and further refine accuracy in dynamic surgical environments.

Summary updated

Abstract

Surgical robots are usually controlled using a priori models based on the robots' geometric parameters, which are calibrated before the surgical procedure. One of the challenges in using robots in real surgical settings is that those parameters can change over time, consequently deteriorating control accuracy. In this context, our group has been investigating online calibration strategies without added sensors. In one step toward that goal, we have developed an algorithm to estimate the pose of the instruments' shafts in endoscopic images. In this study, we build upon that earlier work and propose a new framework to more precisely estimate the pose of a rigid surgical instrument. Our strategy is based on a novel pose estimation model called MBAPose and the use of synthetic training data. Our experiments demonstrated an improvement of 21 % for translation error and 26 % for orientation error on synthetic test data with respect to our previous work. Results with real test data provide a baseline for further research.
BibTeX
@inproceedings{yoshimura2021mbapose,
  author = {Masakazu Yoshimura and Murilo Marques Marinho and Kanako Harada and Mamoru Mitsuishi},
  title = {{MBAPose: Mask and Bounding-Box Aware Pose Estimation of Surgical Instruments with Photorealistic Domain Randomization}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=xNdflqPddG},
}
Figure