Abstract
Image-based navigation is widely considered the next frontier of minimally invasive surgery, promising increased access to reproducible, safe, and high-precision procedures. A critical component of this navigation is 2D/3D registration—estimating the spatial relationship between 3D structures (e.g., CT scans) and 2D intraoperative images (e.g., fluoroscopy). While a mature technique, its clinical transition has been hindered by challenges like optimization brittleness and limited single-view performance. This review examines how machine learning (ML) approximates complex functional mappings to solve these notorious challenges, offering a systematic summary of recent advances and a perspective on future needs.
Introduction
In image-guided interventions, surgeons require precise spatial mapping to understand tool locations relative to patient anatomy. 2D/3D registration serves as the "GPS" for these procedures. Traditionally, this was solved via intensity-based optimization—an iterative process prone to getting stuck in local minima if not initialized perfectly. Furthermore, estimating 3D depth from a single 2D projection is mathematically ill-posed, and traditional methods often struggle with inconsistencies like surgical hardware appearing in the 2D view but not the 3D model.
Methodology
To assess the impact of machine learning on this field, a systematic review was conducted following a rigorous selection process. The authors analyzed the literature to categorize the functional roles ML plays in enhancing registration workflows.
A Taxonomy of Machine Learning Approaches
The review identifies five key functional themes where ML has reshaped 2D/3D registration:
- Contextualization: Utilizing CNNs for landmark detection or anatomy segmentation to automate the "starting guess" (initialization) and mask irrelevant image features.
- Representation Learning: Compressing complex 3D deformations into low-dimensional latent spaces (e.g., via autoencoders), simplifying the optimization landscape.
- Similarity Modeling: Replacing traditional mathematical formulas with learned similarity metrics that offer a wider "capture range" and smoother optimization surfaces.
- Direct Parameter Regression: Employing deep learning to predict 3D poses directly from 2D images in a single forward pass, enabling real-time potential.
- Verification: Training models for quality control to predict registration success or provide uncertainty estimates.
The Data Challenge: Simulation vs. Reality
A significant finding of this review is the field's heavy reliance on synthetic data. Of the 48 studies, 32 relied exclusively on Digitally Reconstructed Radiographs (DRRs) simulated from CT volumes. This creates a "synthetic trap" where models fail to generalize to real clinical environments due to domain shift—the difference between perfect simulations and noisy, high-scatter clinical X-rays.
Clinical Perspective and Future Directions
While ML has improved registration speed and automation, most research focuses on single-object registration. Real-world surgical scenarios are far more complex, involving multiple moving bone fragments, surgical robots, and implants.
To bridge the "bench-to-bedside" gap, the community must prioritize:
- Standardized Benchmarks: Large-scale public datasets with ground-truth 3D poses.
- Robustness: Techniques to handle domain shift between simulation and the OR.
- Compositional Models: Frameworks capable of simultaneous multi-body tracking.
- Uncertainty Quantification: Systems that can signal when an alignment is unreliable, which is critical for surgeon trust in autonomous workflows.
Conclusion
Machine learning has significantly advanced the state of 2D/3D registration, particularly in automation and capture range. However, the transition to routine clinical use requires a shift from synthetic benchmarks to robust, multi-object frameworks validated within real-world surgical workflows.
Summary updated