Two-stage multi-task deep learning framework for simultaneous pelvic bone segmentation and landmark detection from CT images

Haoyu Zhai, Zhonghua Chen, Lei Li, Hairong Tao, Jinwu Wang, Kang Li, Moyu Shao, Xiaomin Cheng, Jing Wang, Xiang Wu, Chuan Wu, Xiao Zhang, Lauri Kettunen, Hongkai WangInternational Journal of Computer Assisted Radiology and Surgery 2023 · DOI

Introduction

Total hip arthroplasty (THA) is a critical surgical intervention for patients suffering from severe hip joint pathologies. The success of THA is contingent upon precise preoperative planning, which requires accurate segmentation of the pelvic bone and identification of anatomical landmarks to ensure optimal prosthesis placement. Traditionally, this process is performed manually, consuming 10–30 minutes per patient and introducing potential for human error.

Existing automated solutions struggle with two primary challenges. First, diseased anatomy—such as osteoarthritis or fractures—often results in "weak" bone boundaries and narrowed joint spaces, which can mislead standard segmentation algorithms. Second, most models treat segmentation and landmark detection as isolated tasks, failing to leverage the inherent spatial relationships between bone structures and their landmarks.

CT images of diseased pelvic cases highlighting the challenges of boundary detection and ROI identification.

Methodology: A Two-Stage Multi-Task Approach

To address these issues, researchers developed a two-stage multi-task cascade network designed to make segmentation and landmark detection mutually reinforcing.

Stage 1: Global Coarse Segmentation and Detection

The first stage operates on low-resolution, down-sampled CT images. A shared encoder extracts features that are then processed by two independent decoders:

  1. Segmentation Decoder: Generates coarse masks for the sacrum and hip bones.
  2. Landmark Decoder: Produces heatmaps for 16 key anatomical landmarks. This stage provides a global context and identifies the specific Region of Interest (ROI) for further refinement.

Stage 2: Local Refined Segmentation with Edge Enhancement

In the second stage, the model crops high-resolution acetabular regions based on the landmarks identified in Stage 1. It employs an edge-enhanced dual-task network. This network simultaneously predicts bone segmentation and bone edges. By integrating an explicit edge-detection branch, the model "learns" to identify subtle gradients in diseased tissue where boundaries are otherwise ambiguous.

The framework utilizes a 3D U-Net backbone with a specialized loss function: Dice Loss for segmentation, Focal Loss for landmarks (addressing class imbalance), and Binary Cross Entropy (BCE) for edge detection.

Results and Performance

The framework was evaluated on a dataset of 81 CT images, including 31 diseased cases (osteonecrosis, osteoarthritis, DDH, and fractures) and 50 healthy cases from the COLONOG dataset.

  • Segmentation Accuracy: The model achieved a Dice Similarity Coefficient (DSC) of 0.94 for the sacrum and 0.97 for the hips. The second stage refinement improved acetabulum segmentation by 5.42% over the coarse stage.
  • Landmark Precision: The average distance error for bone landmarks was reduced to 3.24 mm.
  • Efficiency: The entire automated workflow completes in approximately 10 seconds, a significant improvement over the 10–30 minutes required for manual annotation and nearly twice as fast as standard U-Net implementations.

Conclusion

The proposed two-stage multi-task framework demonstrates that integrating landmark detection and edge-enhancement tasks significantly improves the robustness of pelvic bone segmentation, particularly in complex diseased cases. By automating this workflow with high precision and speed, the model offers a viable path toward more efficient and accurate preoperative planning for hip surgeries. Future work will focus on expanding the dataset and integrating more complex inter-task constraints to further enhance clinical utility.

Summary updated

BibTeX
@article{zhai2023two,
  author = {Haoyu Zhai and Zhonghua Chen and Lei Li and Hairong Tao and Jinwu Wang and Kang Li and Moyu Shao and Xiaomin Cheng and Jing Wang and Xiang Wu and Chuan Wu and Xiao Zhang and Lauri Kettunen and Hongkai Wang},
  title = {{Two-stage multi-task deep learning framework for simultaneous pelvic bone segmentation and landmark detection from CT images}},
  journal = {International Journal of Computer Assisted Radiology and Surgery},
  year = {2023},
  doi = {10.1007/s11548-023-02976-1},
}
Figure