JointNET: A Deep Model for Predicting Active Sacroiliitis from Sacroiliac Joint Radiography

Sevcan Türk, Ahmet Demirkaya, Mehmet Y. Turali, Cenk Hepdurgun, Salman Uh Dar, Ahmet K Karabulut, Aynur Azizova, İpek Tamsel, Üstün Aydıngöz, Mehmet Argın, Tolga Çukur, Cukur, TolgaarXiv 2023 · DOI · arXiv

TL;DR

JointNET is a deep learning ensemble model that detects active sacroiliitis (inflammation of the sacroiliac joint) directly from standard radiographs—a task traditionally considered impossible for human radiologists. By identifying subtle image features invisible to the naked eye, the model achieves over 90% accuracy, potentially offering a low-cost screening alternative to expensive MRI scans.

What Problem Does This Paper Address?

Sacroiliitis is a hallmark of axial spondyloarthritis, a chronic inflammatory disease. Currently, MRI is the "gold standard" for diagnosis because it can detect early-stage inflammation (bone marrow edema) before permanent structural damage occurs. However, MRI is expensive, time-consuming, and not always accessible.

Standard radiographs (X-rays) are the first-line imaging tool because they are cheap and ubiquitous. The problem is that X-rays primarily show "bony damage" that happens late in the disease. In the early, active inflammatory stage, X-rays often look completely normal to human observers (classified as Grade 0). This paper investigates whether deep learning can find "hidden" signatures of active inflammation in these seemingly normal X-rays.

Methods

The researchers developed JointNET, an ensemble convolutional neural network (CNN).

  • Architecture: The model combines two high-performing backbones: DenseNet121 and ResNet152. The outputs of these two networks are aggregated via averaging to produce a final prediction.
  • Multi-modal Input: Beyond just the image, the model incorporates clinical metadata—age and gender—as separate input channels, which the authors found significantly improved performance.
  • Preprocessing: The pipeline includes automated region-of-interest (ROI) extraction via template matching and adaptive histogram equalization to normalize signal intensity variations across different X-ray machines.
  • Training: They used transfer learning (initializing with ImageNet weights) and a 10-fold cross-validation procedure to ensure robust results.

Datasets

The study utilized a retrospective dataset from two university hospitals:

  • Size: 1,537 sacroiliac joints (SIJs) from 768 patients.
  • Selection Criteria: All X-rays were "Grade 0" (normal appearance) according to the modified New York criteria.
  • Ground Truth: Every X-ray was paired with an MRI scan taken within 1.5 months. Of the joints, 330 showed active inflammation on MRI according to ASAS criteria.
  • Augmentation: The dataset was augmented to 1,752 samples to improve training stability.

Key Insights and Ideas

  • The "Invisible" Signal: The most provocative finding is that active inflammation causes "subtle density changes" in the surrounding tissue that are quantitatively detectable by AI even when a radiologist sees nothing.
  • Clinical Context is King: Integrating age and gender wasn't just a minor tweak; it correlates with known clinical patterns where these factors influence the prevalence and presentation of the disease, leading to higher sensitivity.
  • Human vs. Machine Gap: Radiologists spent about 3 seconds per image and achieved accuracies barely better than a coin flip (54-64%). JointNET’s 90% accuracy suggests that for certain diagnostic tasks, the human visual system is simply not the right tool for the job.

Key Results

  • JointNET Performance: The model achieved a mean AUROC of 89.2 and a mean accuracy of 90.2%.
  • Sensitivity/Specificity: JointNET demonstrated a sensitivity of 69.0% and a high specificity of 90.4%.
  • Comparison: Experienced radiologists (with 20+ years of experience) were significantly outperformed. Their sensitivities ranged from 46% to 65%, with specificities between 57% and 64%.
  • Ablation Studies: The authors showed that without data augmentation and normalization, performance dropped significantly (e.g., AUC falling from ~0.82 to ~0.75 in individual backbones).

Limitations and Future Work

  • Scope: The study focused exclusively on Grade 0 SIJs. While this is the most difficult diagnostic category, the model's performance on higher grades of sacroiliitis remains untested in this specific paper.
  • Generalizability: The researchers did not have an external validation set (data from a completely different institution). This is a crucial next step to ensure the model hasn't just "overfit" to the specific imaging protocols of the two hospitals in the study.
  • Clinical Integration: While the results are promising as a "proof of concept," the authors note that further research is needed before radiographs can truly replace MRI for evaluating active inflammation in a clinical setting.

Summary updated

Abstract

Purpose: To develop a deep learning model that predicts active inflammation from sacroiliac joint radiographs and to compare the success with radiologists. Materials and Methods: A total of 1,537 (augmented 1752) grade 0 SIJs of 768 patients were retrospectively analyzed. Gold-standard MRI exams showed active inflammation in 330 joints according to ASAS criteria. A convolutional neural network model (JointNET) was developed to detect MRI-based active inflammation labels solely based on radiographs. Two radiologists blindly evaluated the radiographs for comparison. Python, PyTorch, and SPSS were used for analyses. P<0.05 was considered statistically significant. Results: JointNET differentiated active inflammation from radiographs with a mean AUROC of 89.2 (95% CI:86.8%, 91.7%). The sensitivity was 69.0% (95% CI:65.3%, 72.7%) and specificity 90.4% (95% CI:87.8 % 92.9%). The mean accuracy was 90.2% (95% CI: 87.6%, 92.8%). The positive predictive value was 74.6% (95% CI: 72.5%, 76.7%) and negative predictive value was 87.9% (95% CI: 85.4%, 90.5%) when prevalence was considered 1%. Statistical analyses showed a significant difference between active inflammation and healthy groups (p<0.05). Radiologists accuracies were less than 65% to discriminate active inflammation from sacroiliac joint radiographs. Conclusion: JointNET successfully predicts active inflammation from sacroiliac joint radiographs, with superior performance to human observers.
BibTeX
@misc{turk2023jointnet,
  author = {Sevcan Türk and Ahmet Demirkaya and Mehmet Y. Turali and Cenk Hepdurgun and Salman UH Dar and Ahmet K Karabulut and Aynur Azizova and İpek Tamsel and Üstün Aydıngöz and Mehmet Argın and Tolga Çukur and Cukur, Tolga},
  title = {{JointNET: A Deep Model for Predicting Active Sacroiliitis from Sacroiliac Joint Radiography}},
  year = {2023},
  eprint = {2301.10769},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2301.10769},
}
Figure