Deep Learned Segmentations of Inflammation for Novel ⁹⁹mTc-maraciclatide Imaging of Rheumatoid Arthritis

Robert Cobb, Gary Cook, Andrew J. ReaderDiagnostics 2023 · DOI

TL;DR

This paper introduces the first deep learning approach for segmenting inflammation in rheumatoid arthritis (RA) using novel 99mTc-maraciclatide imaging. The study demonstrates that a nnU-Net model significantly outperforms a traditional thresholding method in accurately identifying normal, low, and highly inflamed tissue, paving the way for AI-assisted clinical workflows in RA management.

What Problem Does This Paper Address?

Rheumatoid arthritis (RA) is a debilitating autoimmune disease affecting joints. Current imaging techniques like ultrasound and MRI are used to monitor RA, but they have limitations. 99mTc-maraciclatide gamma camera imaging offers a promising alternative, capable of visualizing inflammation across all joints in a single scan. However, analyzing these scans to quantify inflammation can be time-consuming and subject to inter-observer variability. This paper aims to address this by developing automated segmentation tools using deep learning to precisely identify and quantify inflamed regions in 99mTc-maraciclatide images, thereby improving diagnostic efficiency and consistency.

Methods

The researchers compared two distinct approaches for segmenting inflammation:

  • Thresholding Model: This served as a baseline. It involved smoothing the input image with a Gaussian kernel and then applying learned lower and upper threshold values to segment normal, low, and high inflammation. This model had a very limited number of trainable parameters (nine in total).
  • nnU-Net Model: This is a state-of-the-art, self-configuring convolutional neural network (CNN) framework specifically designed for biomedical image segmentation. The study utilized the 2D U-Net variant of nnU-Net, employing instance normalization, leaky ReLU activation, and a combination of Dice and cross-entropy loss. Data augmentation techniques like rotations and Gaussian noise were also applied.

Both models were trained and evaluated using a cross-validation strategy to ensure robust performance assessment.

Datasets

The study utilized a dataset of 192 hand and wrist images from 48 RA patients. These images were acquired using a gamma camera after injection of 99mTc-maraciclatide. The dataset was manually segmented by an experienced clinician into non-mutually exclusive regions of normal, low, and high inflammation. For the purpose of training and evaluation, the labels were made mutually exclusive by prioritizing the highest degree of inflammation for any given pixel.

Key Insights and Ideas

  • nnU-Net's Superiority: The nnU-Net model demonstrated a clear advantage over the simpler thresholding method, particularly in segmenting low and high inflammation. This highlights the power of deep learning architectures in capturing complex patterns in medical imaging data.
  • Challenges in Low Inflammation Segmentation: Both models struggled more with segmenting low inflammation compared to high inflammation or normal tissue. This is a common challenge in medical image analysis, often stemming from subtle visual cues and class imbalance.
  • Intraobserver Variability: The study found that even human segmentation exhibits significant variability, especially for low and high inflammation. The nnU-Net model's performance was comparable to this intraobserver variability, suggesting it can achieve a level of consistency close to that of a single human expert.
  • Combined Inflammation Class: When low and high inflammation were combined into a single "inflamed" class, the nnU-Net model showed improved performance, indicating that its ability to detect any level of inflammation is robust.

Key Results

The primary evaluation metric was the modified Dice score, which accounts for empty segmentations.

  • nnU-Net Model: Achieved modified Dice scores of 0.94 ± 0.01 for normal tissue, 0.51 ± 0.14 for low inflammation, and 0.76 ± 0.16 for high inflammation.
  • Thresholding Model: Achieved modified Dice scores of 0.92 ± 0.01 for normal tissue, 0.14 ± 0.07 for low inflammation, and 0.35 ± 0.21 for high inflammation.

For the combined "inflamed" class, the nnU-Net model achieved a modified Dice score of 0.72 ± 0.12, which was comparable to the intraobserver score of 0.76. The ROC curve analysis also showed a higher Area Under the Curve (AUC) for nnU-Net (0.96) compared to the thresholding model (0.88).

Limitations and Future Work

  • Dataset Size and Diversity: The dataset was relatively small (48 patients) and collected from a single center. This limits the generalizability of the findings, and future work should focus on larger, multi-center datasets.
  • Clinician Variability: The study relied on a single clinician's segmentation. Incorporating segmentations from multiple observers would provide a more comprehensive understanding of interobserver variability and allow for training more robust models.
  • Segmentation Metric: The authors acknowledge that the Dice score, while standard, has limitations with empty segmentations and may not perfectly capture clinical relevance, where identifying if a joint is inflamed might be more critical than precise boundary delineation.
  • Imbalanced Data: The dataset exhibits class imbalance, particularly for low inflammation. Future research could explore advanced loss functions (e.g., focal loss, Tversky loss) or more sophisticated data augmentation techniques (e.g., GANs, diffusion models) to address this.
  • Clinical Utility: While promising, this work is a first step. Future research should focus on integrating these AI tools into clinical workflows and validating their impact on patient care and diagnostic efficiency.

Summary updated

Abstract

Rheumatoid arthritis (RA) is an autoimmune disease that causes joint pain, stiffness, and erosion. Power Doppler ultrasound and MRI are imaging modalities used in detecting and monitoring the disease, but they have limitations. ⁹⁹<sup>m</sup>Tc-maraciclatide gamma camera imaging is a novel technique that can detect joint inflammation at all sites in a single examination and has been shown to correlate with power Doppler ultrasound. In this work, we investigate if machine learning models can be used to automatically segment regions of normal, low, and highly inflamed tissue from 192 ⁹⁹<sup>m</sup>Tc-maraciclatide scans of the hands and wrists from 48 patients. Two models were trained: a thresholding model that learns lower and upper threshold values and a neural-network-based nnU-Net model that uses a convolutional neural network (CNN). The nnU-Net model showed 0.94 ± 0.01, 0.51 ± 0.14, and 0.76 ± 0.16 modified Dice scores for segmenting the normal, low, and highly inflamed tissue, respectively, when compared to clinical segmented labels. This outperforms the thresholding model, which achieved modified Dice scores of 0.92 ± 0.01, 0.14 ± 0.07, and 0.35 ± 0.21, respectively. This is an important first step in developing artificial intelligence (AI) tools to assist clinicians' workflow in the use of this new radiopharmaceutical.
BibTeX
@inproceedings{cobb2023deep,
  author = {Robert Cobb and Gary Cook and Andrew J. Reader},
  title = {{Deep Learned Segmentations of Inflammation for Novel ⁹⁹mTc-maraciclatide Imaging of Rheumatoid Arthritis}},
  booktitle = {Diagnostics},
  year = {2023},
  doi = {10.3390/diagnostics13213298},
}
Figure