Abstract
In this paper, we aim to design an automatic camera pose estimation pipeline for clinical spaces such as catheterization laboratories. Our proposed pipeline exploits Scaled-YOLOv4 to detect fixed objects. We adopt the self-supervised key-point detector SuperPoint in combination with SuperGlue, a keypoint matching technique based on graph neural networks. Thus, we match key-points on input images with annotated reference points. Reference points are chosen on fixed objects in the scene, such as corners of door posts or windows. The point-correspondences between the image coordinates and the 3D coordinates are applied to the Perspective-n-Point algorithm to estimate the pose of each camera. Compared with other camera pose estimation methods, the proposed pipeline does not require the construction of 3D point-cloud model of the scene or placing a polyhedron object in the scene before each required calibration. Using videos from real procedures, we show that the pipeline can estimate the camera pose with high accuracy.
Introduction
In high-stakes environments like catheterization laboratories or operating rooms (ORs), tracking the 3D movement of personnel and equipment is vital for workflow analysis and patient safety. However, these rooms are dynamic: cameras are frequently bumped or moved, and equipment is shifted. To maintain accuracy, cameras require regular extrinsic calibration—the process of determining exactly where a camera is located and which way it is pointing in 3D space.
Traditionally, this requires a technician to hold a calibration pattern (like a checkerboard) in front of the camera or involves building a labor-intensive 3D point-cloud model of the room. In a sterile, busy clinical environment, these manual steps are disruptive and often impractical. This research proposes a "set-it-and-forget-it" solution that uses the room’s own geometry to keep cameras calibrated automatically.
Methods
The proposed multi-stage computer vision pipeline treats the room itself as the calibration target. The process is divided into four primary stages:
- Fixed Object Detection: The system uses Scaled-YOLOv4 to identify "anchor" objects that remain static, such as door posts, windows, and wall-mounted monitors.
- Key-Point Detection & Matching: Once an object is localized, SuperPoint identifies specific corners or vertices. These are matched against a pre-recorded database of 3D coordinates using SuperGlue, a graph neural network designed for finding correspondences between different viewpoints.
- Filtering for Robustness: To eliminate "hallucinated" matches, the authors implemented Nearest Centroid Distance (NCD) and Reprojection Error Minimization (RPEM) strategies. These ensure only geometrically consistent points are utilized.
- Pose Estimation: Finally, the 2D image points and their corresponding 3D world coordinates are processed via the EPnP (Efficient Perspective-n-Point) algorithm to calculate the camera’s rotation and translation.
Experimental Setup and Results
The study utilized real-world data recorded at the Reinier de Graaf Gasthuis in Delft, Netherlands. The environment was a catheterization laboratory equipped with five cameras. The team annotated 12 classes of fixed objects across nearly 4,200 image frames. To provide a "ground truth" for evaluation, a 3D cube wireframe with 27 known vertices was placed on the operating table to measure 3D localization accuracy.
The pipeline demonstrated high precision in this challenging environment:
- Object Detection: Scaled-YOLOv4 achieved a Mean Average Precision ([email protected]) of 0.995.
- Accuracy: The best-performing configuration (using the RPEM filter) achieved a reprojection error of 5.79 pixels.
- Physical Precision: The system achieved a Euclidean distance error of just 3.28 cm when averaged across the 27 key-points on the test cube, which is sufficient for tracking surgical workflows.
Key Insights and Discussion
The core innovation lies in utilizing the static "bones" of the OR (doors, windows, vents) as permanent calibration markers. By using YOLOv4 as a pre-filter, the system significantly reduces the search space for key-point matching, minimizing false positives.
While highly effective, the system currently requires an initial manual setup to build the 3D database of reference points. Future work may focus on making the system more generalizable across different hospital environments and addressing potential occlusions when equipment or personnel block the reference objects.
Summary updated