Automatic knee osteoarthritis diagnosis from plain radiographs:a deep learning-based approach

Aleksei Tiulpin, Jérôme Thevenot, Esa Rahtu, Petri Lehenkari, Simo Saarakkala

Introduction

Knee osteoarthritis (OA) is a leading cause of global disability. Despite its prevalence, clinical diagnosis remains highly subjective, typically relying on the Kellgren-Lawrence (KL) scale (0–4) to grade X-ray severity. This manual process is characterized by significant inter-reader variability, where experts often disagree on the same image. Furthermore, the "black box" nature of many deep learning models hinders clinical adoption, as practitioners require transparent justifications for automated diagnoses. This research introduces a Deep Siamese Convolutional Neural Network (CNN) designed to provide human-level accuracy with explainable outputs.

Methodology

The proposed architecture leverages anatomical symmetry through a Deep Siamese CNN approach. Rather than processing a full knee radiograph, the network analyzes two distinct patches: one from the lateral (outer) side and one from the medial (inner) side of the joint. By horizontally flipping the medial patch and using shared weights across both branches, the model is forced to learn consistent feature extraction logic, reducing parameter count and improving robustness.

Overview of the proposed Deep Siamese CNN pipeline, including data pre-processing, model fusion, and explainable output.

To ensure transparency, the system incorporates a modified Gradient-weighted Class Activation Mapping (GradCAM) technique. This generates "attention maps" that highlight specific radiological features, such as osteophytes or joint-space narrowing, which drive the model's classification. The final diagnostic output is derived from an ensemble of three models trained with different random seeds to enhance stability.

Datasets and Validation

The model's generalizability was rigorously tested across independent cohorts. Training was conducted using 18,376 images from the Multicenter Osteoarthritis Study (MOST), while validation was performed on a completely separate set of 5,960 images from the Osteoarthritis Initiative (OAI). This cross-dataset validation demonstrates the model's ability to handle variations in image contrast and positioning across different clinical environments.

Key Results and Insights

The Deep Siamese CNN achieved state-of-the-art performance, demonstrating "excellent agreement" with clinical experts. Key metrics include:

  • Quadratic Kappa Coefficient: 0.83
  • Area Under Curve (AUC): 0.93 for radiographic OA (KL grade ≥ 2)
  • Multi-class Accuracy: 66.71%

Notably, the model outperformed standard ResNet-34 baselines by approximately 4% in the critical KL-2 category, which represents the threshold for early OA diagnosis. Visual analysis of the attention maps confirmed that the AI focuses on the same anatomical regions as radiologists—specifically the tibial and femoral corners where osteophytes typically manifest.

Conclusion and Future Directions

This study demonstrates that anatomy-informed deep learning can provide robust, transferable, and interpretable tools for OA diagnosis. While the current model utilized downsampled 8-bit images, future work may explore high-resolution bone texture analysis. Additionally, further research is required to evaluate performance on non-standardized clinical radiographs where patient positioning may vary. The authors have released their training code and standardized datasets to facilitate further benchmarking in the field.

Summary updated

Abstract

Knee osteoarthritis (OA) is the most common musculoskeletal disorder. OA diagnosis is currently conducted by assessing symptoms and evaluating plain radiographs, but this process suffers from subjectivity. In this study, we present a new transparent computer-aided diagnosis method based on the Deep Siamese Convolutional Neural Network to automatically score knee OA severity according to the Kellgren-Lawrence grading scale. We trained our method using the data solely from the Multicenter Osteoarthritis Study and validated it on randomly selected 3,000 subjects (5,960 knees) from Osteoarthritis Initiative dataset. Our method yielded a quadratic Kappa coefficient of 0.83 and average multiclass accuracy of 66.71\% compared to the annotations given by a committee of clinical experts. Here, we also report a radiological OA diagnosis area under the ROC curve of 0.93. We also present attention maps -- given as a class probability distribution -- highlighting the radiological features affecting the network decision. This information makes the decision process transparent for the practitioner, which builds better trust toward automatic methods. We believe that our model is useful for clinical decision making and for OA research; therefore, we openly release our training codes and the data set created in this study.
BibTeX
@inproceedings{tiulpin2018automatic,
  author = {Aleksei Tiulpin and Jérôme Thevenot and Esa Rahtu and Petri Lehenkari and Simo Saarakkala},
  title = {{Automatic knee osteoarthritis diagnosis from plain radiographs:a deep learning-based approach}},
  booktitle = {University},
  year = {2018},
}
Figure