Interpretable vertebral fracture quantification via anchor-free landmarks localization

Alexey Zakharov, Maxim Pisov, Alim Bukharaev, Alexey Petraikin, Sergey Morozov, Victor Gombolevskiy, Mikhail Belyaev

TL;DR

Researchers have developed a two-step AI pipeline that localizes the spine in 3D and uses anchor-free landmark detection to quantify vertebral fractures. By predicting six specific keypoints per vertebra, the system provides clinically interpretable measurements (the Genant scale) and remains robust even in complex cases like scoliosis, processing a full 3D CT scan in just two seconds.

What Problem Does This Paper Address?

Vertebral compression fractures (VCFs) are critical early indicators of osteoporosis, yet they are frequently missed by radiologists—sometimes at rates exceeding 50%. This happens because CT scans are typically reviewed in axial (top-down) slices, whereas fractures are best seen in the sagittal (side) plane.

While previous AI attempts have automated fracture detection, they face three major hurdles:

  1. Interpretability: Most models provide a "black box" probability score rather than measurements a doctor can verify.
  2. Geometric Complexity: Patients with scoliosis (curved spines) or severe deformities often break standard detection algorithms.
  3. Segmentation Dependency: Many methods rely on perfect bone segmentation, which ironically fails most often on the severely fractured bones the system is meant to find.

Methods

The authors propose a two-stage "localize then quantify" approach:

  • Step 1: Spine Localization & Straightening: The system uses a 3D UNet-like architecture with a soft-argmax layer to find the spinal centerline. It then "straightens" the spine by interpolating the 3D volume along this curve. This transforms a potentially twisted, scoliotic spine into a standardized 2D sagittal image where all vertebrae are aligned vertically.
  • Step 2: Anchor-Free Landmark Localization: Instead of using traditional rectangular bounding boxes (which don't fit the shape of a vertebra well), the model predicts six keypoints for each vertebral body (representing the anterior, middle, and posterior heights).
  • Butterfly-IoU: To handle overlapping predictions during the detection phase, the authors introduced a new metric called "Butterfly-IoU." It calculates the overlap of hexagonal shapes formed by the keypoints rather than standard rectangles, which is far more accurate for deformed bones.
  • Clinical Quantification: Using these keypoints, the system automatically calculates the Genant scale, a clinical standard that compares the ratios of vertebral heights to grade fracture severity.

Datasets

The study utilized a diverse range of data to ensure the model wasn't just "memorizing" one hospital's style:

  • LungCancer500: 100 CT scans from a Moscow screening program, primarily featuring thoracic vertebrae.
  • VerSe-2020: A challenging public benchmark dataset used to test how well the model generalizes to unseen vertebra types and regions.
  • Private Dataset: A large-scale set of 402 chest and abdominal CTs with expert annotations from senior radiologists.
  • Mosmed.ai: An independent test set of 100 studies used for external validation.

Key Insights and Ideas

  • Ditching the Box: The move from anchor-based bounding boxes to anchor-free landmarks is the "aha moment." Vertebrae are organic, irregular shapes; forcing them into rectangles introduces noise. Landmarks allow the model to focus on the specific points required for medical diagnosis.
  • Straightening as Pre-processing: By mathematically "un-curving" the spine first, the researchers simplified a complex 3D problem into a much easier 2D detection task. This makes the model significantly more robust to patient positioning and anatomy.
  • Verifiable AI: Because the output is a set of points and height ratios, a radiologist can look at the screen, see exactly where the model placed the points, and confirm the measurement with a digital ruler. This builds much more trust than a simple "Fracture: 85%" notification.

Key Results

  • High Accuracy: The model achieved a patient-level ROC AUC of up to 0.96 for fracture identification.
  • Speed: The entire pipeline processes a 3D CT volume in approximately 2 seconds on a single GPU, making it viable for real-time clinical use.
  • Precision: Vertebra detection reached a precision and recall of 0.99 on the primary dataset.
  • Generalizability: When tested on the VerSe dataset (without any fine-tuning), the model maintained a high ROC AUC of 0.95, proving it can handle different types of CT scanners and patient populations.

Limitations and Future Work

Despite the strong results, the authors noted some "edge case" failures:

  • Atypical Anatomy: The model occasionally struggled with the C2 vertebra (in the neck) because its shape is significantly different from the thoracic and lumbar vertebrae the model saw most often.
  • Image Artifacts: High levels of noise or metal artifacts (like spinal implants) can still lead to false negatives.
  • Future Directions: The authors suggest that training on even larger, more anatomically diverse datasets (specifically including more cervical and lower lumbar cases) will further improve the system's robustness across the entire spinal column.

Summary updated

Abstract

Vertebral body compression fractures are early signs of osteoporosis. Though these fractures are visible on Computed Tomography (CT) images, they are frequently missed by radiologists in clinical settings. Prior research on automatic methods of vertebral fracture classification proves its reliable quality; however, existing methods provide hard-to-interpret outputs and sometimes fail to process cases with severe abnormalities such as highly pathological vertebrae or scoliosis. We propose a new two-step algorithm to localize the vertebral column in 3D CT images and then detect individual vertebrae and quantify fractures in 2D simultaneously. We train neural networks for both steps using a simple 6-keypoints based annotation scheme, which corresponds precisely to the current clinical recommendation. Our algorithm has no exclusion criteria, processes 3D CT in 2 seconds on a single GPU, and provides an interpretable and verifiable output. The method approaches expert-level performance and demonstrates state-of-the-art results in vertebrae 3D localization (the average error is 1 mm), vertebrae 2D detection (precision and recall are 0.99), and fracture identification (ROC AUC at the patient level is up to 0.96). Our anchor-free vertebra detection network shows excellent generalizability on a new domain by achieving ROC AUC 0.95, sensitivity 0.85, specificity 0.9 on a challenging VerSe dataset with many unseen vertebra types.
BibTeX
@article{zakharov2023interpretable,
  author = {Alexey Zakharov and Maxim Pisov and Alim Bukharaev and Alexey Petraikin and Sergey Morozov and Victor Gombolevskiy and Mikhail Belyaev},
  title = {{Interpretable vertebral fracture quantification via anchor-free landmarks localization}},
  journal = {Medical Image Analysis},
  year = {2023},
  url = {https://openreview.net/forum?id=w4nkM1GRo},
}

Loading related content...

Figure