Clinical super-resolution computed tomography of bone microstructure: application in musculoskeletal and dental imaging

S.j.o. Rytky, Aleksei Tiulpin, Mikko Finnilä, Sakari S. Karhula, Annina Sipola, Väinö Kurttila, Maarit Valkealahti, Petri Lehenkari, Antti Joukainen, Heikki Kröger, Rami K. Korhonen, Simo Saarakkala, Jaakko NiinimäkibioRxiv 2023 · DOI

TL;DR

This paper introduces a deep learning-based super-resolution (SR) method to significantly enhance the spatial resolution of clinical cone-beam CT (CBCT) images, enabling better visualization of bone microstructure. This advancement holds promise for earlier and more precise diagnosis of musculoskeletal and dental conditions, potentially improving patient outcomes.

What Problem Does This Paper Address?

Clinical cone-beam computed tomography (CBCT) is a valuable tool for imaging bone, but its resolution is limited to approximately 0.5 mm. This resolution is insufficient to visualize the fine details of bone microstructure, which are crucial for understanding conditions like osteoarthritis, osteoporosis, and fracture risk, as well as for detailed dental diagnostics and treatment planning. Existing methods for improving resolution are either limited by access to raw data or struggle with artifacts. This paper aims to bridge this gap by developing a widely applicable deep learning approach to boost CBCT image resolution, making microstructural details clinically accessible.

Methods

The authors developed a deep learning-based super-resolution (SR) model inspired by existing architectures but modified with "resize-convolution" layers instead of transposed convolutions. The model was designed to achieve a 4x magnification factor. They explored three different loss function combinations: a baseline model using Mean Squared Error (MSE) and Total Variation (TV), a "structure model" focusing on the Structure Similarity Index (SSIM) to capture microstructure, and a "visual model" combining Mean Absolute Error (MAE), TV, and perceptual loss. Training was performed using the Adam optimizer with four-fold cross-validation. For inference, predictions were combined using a sliding window approach with Gaussian kernel focusing.

Datasets

Training Data: * Twelve knee tissue block samples from five cadavers and six total knee arthroplasty (TKA) patients. * Fifteen human teeth from nine patients.

Ex Vivo Test Set: * 53 osteochondral samples from nine TKA patients and two deceased cadavers.

Clinical Data: * CBCT scans of wrist, knee, ankle, and maxillofacial regions from healthy volunteers and patients. * Preoperative dental CBCT scans. * A commercially available CT quality assurance phantom.

Key Insights and Ideas

  • Leveraging Lab Data for Clinical Gains: Training SR models on high-resolution laboratory micro-CT data from cadaveric samples can effectively enhance the resolution of lower-resolution clinical CBCT images.
  • Structure Matters: The "structure model," specifically designed to capture microstructural details, showed the best performance in predicting bone morphology on unseen ex vivo data, outperforming interpolation and conventional segmentation.
  • Averaging Doesn't Always Help: While averaging predictions across orthogonal planes improved visual quality in some cases, it could smear details crucial for morphometric analysis, especially in smaller samples.
  • Domain Shift Resilience: The SR method demonstrated robustness to domain shift, generalizing well from laboratory training data to diverse clinical datasets without significant performance degradation.

Key Results

  • Improved Bone Microstructure Prediction: The 2D structure model achieved the highest Pearson correlation (rBvTv = 0.817±0.005) for predicting bone morphological parameters on the ex vivo test set, outperforming interpolation (rBvTv = 0.64) and conventional segmentation (rBvTv = 0.67).
  • Enhanced Spatial Resolution: Phantom analysis confirmed a statistically significant increase in spatial resolution for SR-enhanced images, particularly in higher line pair patterns (third, fourth, and fifth).
  • Visual Quality Improvement: Musculoskeletal CBCT images showed more detailed structures with SR predictions compared to interpolation. Dental SR imaging allowed visualization of smaller mineralized structures.
  • Reader Study Nuances: While SR generally improved visualization, reader assessments showed mixed results. A dental radiologist slightly preferred original CBCT images, while a dental surgeon scored one SR model higher. Overall scores were modest, possibly due to the loss of high dynamic range from the original 12-bit CBCT images.
  • Artefacts Noted: Some artefacts were observed, particularly near tooth crowns in dental imaging and within enamel, which could be confused with lesions.

Limitations and Future Work

  • Limited Dynamic Range: The SR models currently lose the high dynamic range of original 12-bit CBCT images, impacting the ability to study Hounsfield unit values. Future work could involve training on higher dynamic range data.
  • Dental Dataset Size: The current dataset of extracted teeth is considered small and not optimal for training SR models specifically for dental applications. A larger maxillofacial dataset is recommended.
  • Perpendicular Plane Resolution: The 2D models did not fully account for changes in the perpendicular plane. Future improvements could involve a three-channel input image including adjacent slices.
  • Qualitative Clinical Assessment: Most clinical comparisons were qualitative. Future studies could involve blinded reader studies with more radiologists to assess diagnostic image quality.
  • Post-processing vs. Reconstruction: The SR was applied as post-processing. Integrating SR directly into the reconstruction domain, as some vendors are starting to do, could be a more seamless approach.
  • Potential for "Hallucination": Generative adversarial networks (GANs), while not the primary focus here, could pose a higher risk of generating non-existent pathological features compared to the CNN-based approach used.
  • Artefacts in Dental Imaging: Artefacts in enamel could be confused with caries. Specialized training datasets with preclinical scans of entire jawlines and soft tissues, or animal models, are suggested to improve dental SR.

Summary updated

Abstract

Abstract Objectives Clinical cone-beam computed tomography (CBCT) devices are limited to imaging features of half a millimeter in size. Hence, they do not allow clinical quantification of bone microstructure, which plays an important role in osteoarthritis, osteoporosis and fracture risk. For maxillofacial imaging, changes in small mineralized structures are important for dental, periodontal and ossicular chain diagnostics as well as treatment planning. Deep learning (DL)-based super-resolution (SR) models could allow for better evaluation of these microstructural details. In this study, we demonstrate a widely applicable method for increasing the spatial resolution of clinical CT images using DL, which only requires training on a limited set of data that are easy to acquire in a laboratory setting from e.g. cadaver knees. Our models are assessed rigorously for technical image quality, ability to predict bone microstructure, as well as clinical image quality of the knee, wrist, ankle and dentomaxillofacial region. Materials and methods Knee tissue blocks from five cadavers and six total knee replacement patients as well as 14 extracted teeth from eight patients were scanned using micro-computed tomography. The images were used as training data for the developed DL-based SR technique, inspired by previous studies on single-image SR. The technique was benchmarked with an ex vivo test set, consisting of 52 small osteochondral samples imaged with clinical and laboratory CT scanners, to quantify bone morphometric parameters. A commercially available quality assurance phantom was imaged with a clinical CT device, and the technical image quality was quantified with a modulation transfer function. To visually assess the clinical image quality, CBCT studies from wrist, knee, ankle, and maxillofacial region were enhanced with SR and contrasted to interpolated images. A dental radiologist and dental surgeon reviewed maxillofacial CBCT studies of nine patients and corresponding SR predictions. Results The SR models yielded a higher Pearson correlation to bone morphological parameters on the ex vivo test set compared to the use of a conventional image processing pipeline. The phantom analysis confirmed a higher spatial resolution on the images enhanced by the SR approach. A statistically significant increase of spatial resolution was seen in the third, fourth, and fifth line pair patterns. However, the predicted grayscale values of line pair patterns exceeded those of uniform areas. Musculoskeletal CBCT images showed more details on SR predictions compared to interpolation. Averaging predictions on orthogonal planes improved visual quality on perpendicular planes but could smear the details for morphometric analysis. SR in dental imaging allowed to visualize smaller mineralized structures in the maxillofacial region, however, some artifacts were observed near the crown of the teeth. The readers assessed mediocre overall scores in all categories for both CBCT and SR. Although not statistically significant, the dental radiologist slightly preferred the original CBCT images. The dental surgeon scored one of the SR models slightly higher compared to CBCT. The interrater variability κ was mostly low to fair. The source code ( https://doi.org/10.5281/zenodo.8041943 ) and pretrained SR networks ( https://doi.org/10.17632/4xvx4p9tzv.1 ) are publicly available. Conclusions Utilizing experimental laboratory imaging modalities in model training could allow pushing the spatial resolution limit beyond state-of-the-art clinical musculoskeletal and dental CBCT imaging. Implications of SR include higher patient throughput, more precise diagnostics, and disease interventions at an earlier state. However, the grayscale distribution of the images is modified, and the predictions are limited to depicting the mineralized structures rather than estimating density or tissue composition. Finally, while the musculoskeletal images showed promising results, a larger maxillofacial dataset would be recommended for training SR models in dental applications.
BibTeX
@misc{rytky2023clinical,
  author = {S.J.O. Rytky and Aleksei Tiulpin and Mikko Finnilä and Sakari S. Karhula and Annina Sipola and Väinö Kurttila and Maarit Valkealahti and Petri Lehenkari and Antti Joukainen and Heikki Kröger and Rami K. Korhonen and Simo Saarakkala and Jaakko Niinimäki},
  title = {{Clinical super-resolution computed tomography of bone microstructure: application in musculoskeletal and dental imaging}},
  year = {2023},
  doi = {10.1101/2023.06.28.544314},
  howpublished = {bioRxiv},
}
Figure