Semantic edge-guided 2D/3D registration for vertebrae in radiographs

Ao Shen, Yue Shen, Junfeng Jiang, Zhengming Chen, Rui Huang, Kunjin He, Jie ChenJournal of Image and Graphics 2025 · DOI

TL;DR

This paper introduces a novel 2D/3D registration method for spinal surgery robots that uses semantic edges—specifically the pedicles and vertebral sides—to align pre-operative CT scans with intra-operative X-rays. By employing a lightweight Vision Transformer (ViT) architecture called SCE-UNETR and an anatomically-inspired spacing constraint loss, the authors achieved sub-millimeter accuracy and sub-0.1° rotation error in approximately 5 seconds.

What Problem Does This Paper Address?

In robotic spine surgery, the robot needs to know the exact 3D position of the patient's vertebrae in real-time. This is achieved through 2D/3D registration: aligning a high-resolution pre-operative 3D CT scan with 2D X-ray images taken during the procedure.

Current methods struggle with several issues:

  1. Landmark Sensitivity: Traditional landmark-based methods are often slow and prone to errors if the landmarks are obscured by overlapping tissues.
  2. Intensity Bottlenecks: Intensity-based methods (comparing pixel values directly) have a small "capture range," meaning they fail if the initial guess of the position is even slightly off.
  3. Edge Clutter: Standard edge-detection picks up every physical boundary. In a 2D X-ray, vertebrae overlap significantly, creating a "mess" of edges that confuses registration algorithms.

The authors set out to find a feature set that is robust to noise and overlap while remaining computationally efficient enough for the operating room.

Methods

The researchers proposed a two-stage approach centered on Semantic Edge Extraction.

  • Feature Selection: Instead of using all physical edges, they focus on "semantic" features: the contours of the pedicles and the lateral walls of the vertebral body. These features are visually distinct and less likely to be "muddied" by overlapping spinal joints.
  • SCE-UNETR Architecture: They developed the Spacing Constrained and Efficient UNet TRansformer. This model uses a Vision Transformer (ViT) backbone for global context but replaces standard, heavy convolutions with a Group Multi-axis Hadamard Product Attention (GHPA) module. This change reduces the quadratic complexity of standard attention to linear complexity, making the model much lighter.
  • Spacing Constraint (SC) Loss: The authors incorporated anatomical "prior knowledge" into the training. Since vertebrae have predictable gaps between them, they added a loss term that penalizes the network if the predicted edges imply an anatomically impossible distance between vertebral centers.
  • Registration Pipeline: Once edges are extracted from both the X-ray and the CT-generated projections (DRRs), they use the CMA-ES (Covariance Matrix Adaptation Evolution Strategy) algorithm to iteratively minimize the distance between these edges until the 3D volume perfectly matches the 2D image.

Datasets

The study utilized a mix of simulated and clinical data:

  • Simulated Data: 11 clinical CT scans were used to generate 210 Digitally Reconstructed Radiographs (DRRs) with random pose variations to train the SCE-UNETR model.
  • Real-World Evaluation: The method was validated using clinical X-ray data from the Changzhou First People's Hospital, focusing on the L4 and L5 lumbar vertebrae.

Key Insights and Ideas

  • Semantic > Physical: The core "aha" moment is that "less is more." By ignoring the noisy, overlapping edges of the spinal joints and focusing only on the pedicles and lateral walls, the registration algorithm has a much cleaner "map" to follow, preventing it from getting stuck in local optima.
  • Anatomy as a Regularizer: Deep learning models often make "unnatural" mistakes. By forcing the network to respect the physical spacing between vertebrae (via the SC Loss), the authors significantly improved the accuracy of multi-vertebrae segmentation.
  • Efficiency via Hadamard Products: The use of GHPA allowed the authors to cut the parameter count nearly in half compared to standard UNETR models (348M vs 672M parameters) without sacrificing performance—a critical requirement for real-time surgical applications.

Key Results

The proposed method outperformed state-of-the-art models like SwinUNETR and TransUNet across several metrics:

  • Precision: After pose refinement, the translation error was less than 1 mm and the rotation error was less than 0.1°.
  • Speed: The entire registration process takes roughly 5 seconds, meeting the real-time demands of clinical surgery.
  • Robustness: In tests on real clinical X-rays (which are often noisier than simulations), the method maintained high structural similarity (SSIM) and normalized cross-correlation (NCC) scores, proving it can handle the "reality gap."

Limitations and Future Work

While highly effective, the authors noted two primary areas for improvement:

  1. Two-Stage Process: Currently, the workflow is split into a coarse registration (edge-based) and a fine refinement (intensity-based). The authors aim to develop a more unified, end-to-end network in the future.
  2. Domain Gap: There is still a slight performance drop when moving from synthetic DRR training data to real intra-operative X-rays. Future work will likely involve domain adaptation techniques to make the model even more "at home" with real-world medical imaging noise.

Summary updated

Abstract

目的基于影像引导的脊柱手术机器人系统中,2D/3D配准指的是将术前计算机断层扫描影像与术中X线片配准,用于实现手术机器人对于人体组织的精准空间定位。常见做法是先用标志点进行粗配准,再用灰度法修正位姿。标志点配准问题在于标志点识别精度不高且识别效率较低,灰度法的捕获范围小且对初始位姿敏感。由于脊柱关节边缘重叠且术中X线片图像质量较低,故利用物理边缘作为特征进行2D/3D配准精度不高。因此,提出一种基于语义边缘提取的2D/3D配准方法。方法首先,提取X线片中成像清晰的椎弓根边缘和椎体两侧边缘作为语义特征进行2D/3D配准;同时,面向边缘提取任务,研究一种间距约束的高效“U”形变形器网络,该深度学习网络提高分割效率的同时,保持了边缘分割的准确性,并加入了椎骨间距约束损失的先验信息,进一步提升了多椎骨语义边缘提取的精度。结果模拟数据与真实数据上评估结果表明,本文方法在配准精度与效率方面均优于现有方法;位姿修正后,本文方法平移误差小于1 mm,旋转误差小于0.1°,配准耗时在5 s左右,能较好满足实际临床需求。结论本文提出的基于椎骨语义边缘的2D/3D粗配准方法有效缩小了后续精配准过程的搜索空间,从而提高了配准精度。在边缘提取方面,将哈达玛乘积代替卷积操作的方式以及加入椎骨间距约束损失,提高了语义边缘的提取效率和精度。因此,本文方法能够较好满足2D/3D配准的精度与实时性需求。
BibTeX
@article{shen2025semantic,
  author = {Ao Shen and Yue Shen and Junfeng Jiang and Zhengming Chen and Rui Huang and Kunjin He and Jie Chen},
  title = {{Semantic edge-guided 2D/3D registration for vertebrae in radiographs}},
  journal = {Journal of Image and Graphics},
  year = {2025},
  doi = {10.11834/jig.240001},
}
Figure