Spinal Osteophyte Detection Via Robust Patch Extraction on Minimally Annotated X-Rays

Soumya Snigdha Kundu, Yuanhan Mo, Nicharee Srikijkasemwat, Bartlomiej W. Papiez

TL;DR

This paper introduces SegPatch, a novel automated method for detecting spinal osteophytes in X-rays. By intelligently segmenting vertebrae and expanding contours to capture potential osteophyte locations, SegPatch significantly outperforms traditional tiling methods, achieving 84.5% accuracy even with limited annotations. This work paves the way for more efficient and accurate clinical diagnosis of osteoarthritis.

What Problem Does This Paper Address?

Osteoarthritis (OA) is a common condition characterized by the formation of osteophytes, which are small bony outgrowths. These osteophytes play a crucial role in determining OA severity and can cause significant pain and complications by impinging on nerves. While Magnetic Resonance Imaging (MRI) is often used, X-rays are a more accessible and cost-effective modality. However, detecting these tiny osteophytes in 2D X-rays is challenging due to image quality and the subtle nature of the growths. Manual identification by clinicians is time-consuming, prone to errors, and requires specialized expertise. Existing automated methods have limitations, such as class imbalance issues when processing entire scans or reliance on multi-modality approaches. This paper aims to address the gap in automated spinal osteophyte detection using minimally annotated X-ray data.

Methods

The core innovation of this paper is the SegPatch pipeline, designed to generate more informative patches for osteophyte detection compared to standard tiling approaches.

  1. Vertebrae Segmentation: SegPatch begins by using a deep learning model (VertXNet [9]) to segment individual vertebrae within the spinal X-ray.
  2. Contour Expansion: Instead of simply using the segmented vertebrae, SegPatch expands the contours of these segmented vertebrae. This expansion is crucial because osteophytes often occur at the corners of vertebrae, and a simple segmentation mask might not fully encompass them. The expansion is guided by the observation that osteophytes are typically found at the corners, with specific directions of expansion (-X and +Y) to account for spinal curvature and potential gaps.
  3. Patch Extraction: The expanded contours define the regions from which patches are extracted. This targeted approach focuses on areas likely to contain osteophytes, reducing noise and improving the signal-to-noise ratio for the classifier.
  4. Patch Classification: These generated patches are then fed into a standard classifier (fine-tuned DenseNet-121 models were used) for binary classification: "osteophyte present" or "osteophyte absent."

This method contrasts with the tiling approach, which simply divides the entire X-ray image into a grid of equal-sized, non-overlapping patches. While tiling is a common pre-processing step for object detection, it often leads to a severe class imbalance (many more "absent" patches than "present" ones) and can miss small objects if they fall on patch boundaries.

Datasets

The study utilizes the Second National Health and Nutrition Examination Survey (NHANES II) dataset, collected between 1976-1980. This dataset comprises digitized X-ray films, specifically:

  • 9670 cervical X-ray images (1462 x 1755 pixels)
  • 7430 lumbar X-ray images (2048 x 2487 pixels)

Crucially, only a small subset of these images was annotated:

  • 241 cervical X-ray images
  • 232 lumbar X-ray images

Annotations for osteophytes were provided as pixel coordinates, with each vertebra delineated by six pixel points. The data was split into a 75:25 ratio for training and testing. The paper notes that this is a "minimally annotated" dataset, highlighting the challenge SegPatch aims to overcome.

Key Insights and Ideas

  • The power of targeted patch generation: The core insight is that simply tiling an image creates a lot of noise and class imbalance. By intelligently focusing on regions around the vertebrae (where osteophytes are known to occur), SegPatch generates much more informative patches for the classifier.
  • Contour expansion is key for small object detection: Osteophytes are tiny. Expanding the segmented vertebral contours, rather than just using the exact segmentation, is essential to ensure these small growths are actually included within the extracted patches.
  • Minimally annotated data is viable: The success of SegPatch demonstrates that robust detection models can be trained even with limited manual annotations, which is a significant advantage for clinical applications where annotation is costly and time-consuming.
  • Saliency maps confirm model focus: Using SS-CAM, the authors showed that the classifier, when presented with SegPatch-generated patches, predominantly focuses on the edges of the vertebrae where osteophytes are present, validating the method's approach.

Key Results

  • Superior Accuracy: SegPatch achieved an average test accuracy of 84.5%, outperforming the baseline tiling method by 9.5%.
  • Model Performance: When using a fine-tuned DenseNet-121 model, SegPatch achieved an accuracy score of 84.5% and a specificity score of 86.62%.
  • Patch Distribution: SegPatch generated a more balanced set of patches (1680 "present" vs. 1517 "absent") compared to tiling, which resulted in a highly imbalanced set (1453 "present" vs. 32160 "absent").
  • Off-the-shelf Detector Limitations: Initial experiments with an off-the-shelf detector (FasterRCNN) showed very poor performance, with mAP and accuracy scores as low as 0.1% and 0.2% for smaller bounding box sizes, underscoring the need for a specialized approach.

Limitations and Future Work

  • Patch Size Variability: SegPatch generates non-uniformly sized patches. While this was necessary to avoid missing osteophytes or creating excessive overlaps, it can lead to lower resolution patches for larger areas, which might impact performance. The authors explored uniform patch sizes but found it infeasible.
  • Artifacts and Curvature: The paper acknowledges challenges with image artifacts (like text on scans) and extreme spinal curvature, which can make labeling difficult and impact segmentation accuracy.
  • Future Work: The authors suggest that future methodologies should place greater emphasis on the identification aspect rather than localization, given that models already understand typical osteophyte locations. They also propose that this single-modality approach (X-ray) is efficient and can be as effective as multi-modality methods. The work lays the groundwork for more effective clinical approaches to detecting small structures in X-rays.

Summary updated

BibTeX
@inproceedings{kundu2024spinal,
  author = {Soumya Snigdha Kundu and Yuanhan Mo and Nicharee Srikijkasemwat and Bartlomiej W. Papiez},
  title = {{Spinal Osteophyte Detection Via Robust Patch Extraction on Minimally Annotated X-Rays}},
  booktitle = {ISBI},
  year = {2024},
  doi = {10.1109/isbi56570.2024.10635802},
}

Loading related content...

Figure