Introduction
In the fields of robotics and computer-assisted surgery, high-precision 3D pose estimation—determining an object's exact position and orientation—is critical. While marker-less tracking is a growing area of research, physical markers remain the gold standard for tasks requiring sub-millimeter accuracy, such as robotic grasping or surgical navigation. However, classical marker tracking algorithms are often fragile, failing under suboptimal lighting or significant motion blur.
MarkerPose addresses these challenges by integrating deep learning into a stereo vision pipeline, offering a system that is both faster and more robust than traditional heuristic-based methods.
The Planar Target
The system utilizes a specific planar target design featuring three concentric circles. This geometric arrangement allows the system to calculate a full 6-degree-of-freedom (6-DoF) pose by identifying and distinguishing between the three unique points.
The MarkerPose Pipeline
MarkerPose employs a hierarchical "detect-then-zoom" strategy to achieve real-time performance (up to 62 FPS) without sacrificing precision. The process is divided into two main stages.
1. Coarse Detection
In the first stage, a modified SuperPoint network processes low-resolution versions of the stereo images. It identifies the centers of the three circles at a pixel level and classifies them ($c_0, c_1$, or $c_2$). This ID-tagging is essential for determining the target's orientation.
2. Sub-pixel Refinement and Segmentation
Once the general locations are known, the system crops 120 x 120 pixel patches from the original high-resolution images. These patches are fed into EllipSegNet, a novel, lightweight (1.1 MB) encoder-decoder network.
Unlike systems that attempt to regress coordinates directly, EllipSegNet predicts a binary mask of the ellipse. By using segmentation, the system allows every pixel in the shape to contribute to the center estimation, making it significantly more resilient to noise and blur. Standard geometric fitting is then applied to these masks to find the centers with sub-pixel accuracy.
Results and Robustness
The researchers validated MarkerPose using a high-precision EPSON C3 robot arm (accurate to 0.02 mm) as the ground truth. The system achieved a Mean Absolute Error (MAE) of just 0.046 mm for displacement and 0.032° for rotation.
Beyond accuracy, the system demonstrated remarkable robustness:
- Low Light: Maintained high precision even when image brightness was reduced to 6% of the original.
- Motion Blur: Successfully tracked targets under heavy synthetic motion blur.
- Real-world Application: The system was used to track an ultrasound probe for 3D reconstruction, measuring a submerged cylinder with an absolute error of only 0.18 mm.
Conclusion
MarkerPose demonstrates that deep learning can overcome the brittleness of classical vision systems in high-stakes environments. By combining efficient coarse detection with specialized sub-pixel segmentation, it provides a reliable, real-time solution for demanding medical and industrial applications.
Paper Citation: Meza, J., Romero, L. A., & Marrugo, A. G. (2021). MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
Summary updated