Spatial-Division Augmented Occupancy Field for Bone Shape Reconstruction from Biplanar X-Rays

Jixiang Chen, Yiqun Lin, Haoran Sun, Xiaomeng Li

TL;DR

This paper introduces SdAOF, a novel deep learning framework for reconstructing 3D bone shapes from just two X-ray images. By combining a memory-efficient continuous occupancy field representation with a new "spatial-division augmented distillation" technique to handle X-ray occlusions, SdAOF achieves state-of-the-art results in fine-scale bone reconstruction.

What Problem Does This Paper Address?

Reconstructing detailed 3D bone anatomy is vital for medical applications like surgical planning and implant design. While CT scans provide excellent 3D data, they involve significant radiation exposure and are costly. X-rays offer a more accessible alternative, but extracting precise 3D information from just two views (biplanar X-rays) is a challenging, ill-posed problem. Existing deep learning methods often rely on memory-intensive voxel representations, limiting reconstruction resolution. Furthermore, they struggle to effectively handle the inherent occlusion effects in X-ray images, where one part of the bone might be hidden by other tissues or bone structures.

Methods

The core of SdAOF is the Occupancy Field (OF) representation. Instead of a discrete voxel grid, it uses a continuous function that predicts whether any given 3D point is inside or outside the bone surface. This implicit representation is memory-efficient and allows for flexible reconstruction at different resolutions without retraining.

To tackle the occlusion problem, SdAOF introduces a novel Spatial-Division Augmented Distillation strategy. This involves:

  1. Spatial Division: The 3D reconstruction space is divided into several "subspaces" along the X-ray view direction.
  2. Augmented X-rays: For each subspace, a "Digitally Reconstructed Radiography" (DRR) technique is used to generate an augmented X-ray image that highlights information within that specific subspace.
  3. Teacher-Student Distillation: A "teacher" OF network is trained on these augmented X-rays to learn subspace-specific features and occlusion relationships. This learned knowledge is then "distilled" into a "student" SdAOF network, which processes the original X-ray images. This distillation process guides the student network to better understand and compensate for occlusions in the input X-rays.

The final SdAOF network combines features from the original X-ray (using a student extractor) and the distilled features from the teacher network to predict the occupancy field.

Datasets

The primary dataset used for training and evaluation is the Pelvis dataset from the XrayTo3D benchmark. The authors also mention generating 2D X-ray images using DRRs from the TIGRE package for their experiments.

Key Insights and Ideas

  • Continuous Occupancy Fields are Key: Moving away from voxels to continuous implicit representations is crucial for memory efficiency and high-resolution reconstruction in medical imaging.
  • Occlusion is a Major Hurdle: X-ray images are inherently prone to occlusions, and simply extracting features from raw images is insufficient. Explicitly modeling occlusion relationships is vital for accurate reconstruction.
  • Spatial Division for Occlusion Understanding: Dividing the reconstruction space and generating augmented views for specific subspaces provides a structured way to learn about occlusion patterns.
  • Knowledge Distillation for Feature Enhancement: Using a teacher-student framework to transfer learned occlusion information from augmented views to the student network processing original X-rays is an effective way to improve feature robustness.
  • Scalability is a Game-Changer: The implicit nature of occupancy fields allows for reconstruction at arbitrary resolutions without retraining, a significant advantage over fixed-resolution voxel methods.

Key Results

  • SdAOF outperforms state-of-the-art voxel-based methods on the Pelvis dataset, achieving a substantial reduction in reconstruction error (e.g., ~6% lower Chamfer Distance compared to the previous leading method, 3DReconNet, for the left pelvis).
  • The method demonstrates improved accuracy for the left pelvis, which is more prone to occlusion by the right pelvis.
  • SdAOF is scalable to higher resolutions (e.g., 336³), showing further accuracy improvements (e.g., 2.465 mm to 2.335 mm CD on the left pelvis) while maintaining reasonable inference times (around 11 seconds per sample).
  • Ablation studies confirm the effectiveness of both the spatial-division augmentation and the distillation strategy, with the full SdAOF framework achieving the best results.
  • The method shows robustness to minor variations in imaging parameters like view angles and patient center locations.
  • GPU memory usage is significantly lower compared to voxel-based methods, especially at higher resolutions, as shown in Figure 4.

Limitations and Future Work

  • Limitations: The paper acknowledges that the current method assumes ideal bi-planar view directions and iso-center patients, and real-world deviations can affect magnification. While SdAOF shows robustness, extreme deviations might still pose challenges.
  • Future Work: The authors suggest exploring more generalized methods, such as incorporating adaptive subspace division based on 3D semantics. They also propose extending the framework to more general multi-view scenarios beyond just biplanar inputs, offering a trade-off between radiation dose and accuracy.

Summary updated

BibTeX
@inproceedings{chen2024spatial,
  author = {Jixiang Chen and Yiqun Lin and Haoran Sun and Xiaomeng Li},
  title = {{Spatial-Division Augmented Occupancy Field for Bone Shape Reconstruction from Biplanar X-Rays}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2024},
  doi = {10.1007/978-3-031-72104-5\_64},
}

Loading related content...

Figure