Abstract
Knee osteoarthritis (OA) is a leading cause of disability worldwide, yet its radiographic assessment remains subjective and time-intensive. We present KneeXNeT, an ensemble-based deep learning framework designed to automate Kellgren-Lawrence (KL) grading from X-ray images. KneeXNeT fuses ten state-of-the-art architectures through a shallow neural network meta-learner. To address significant class imbalance, a weighted sampling strategy was employed. The resulting framework achieved a classification accuracy of 72%, surpassing individual models. Furthermore, visual explanations generated via Smooth-GradCAM++ demonstrate that the model focuses on clinically relevant features, such as joint space narrowing, ensuring its decisions are interpretable for clinicians.
Introduction
Radiologists typically assess OA severity using the Kellgren-Lawrence (KL) system, a five-point scale ranging from Grade 0 (None) to Grade 4 (Severe). However, this manual process is prone to inter-observer variability, particularly in the early "Doubtful" (Grade 1) stages where radiographic changes are subtle. While automated AI solutions have been proposed, they frequently struggle with severe class imbalance—where healthy images far outnumber severe cases—and a lack of interpretability. KneeXNeT addresses these challenges by combining a robust ensemble approach with visual "heatmaps" to explain its diagnostic reasoning.
Methods
The development of KneeXNeT involved a multi-stage pipeline:
- Baseline Architectures: Ten pre-trained deep learning models (including ResNet, VGG, DenseNet, MobileNet, EfficientNet, and GoogLeNet) were fine-tuned on knee X-ray data.
- Weighted Sampling: To mitigate the impact of lopsided data, a weighted sampling strategy was implemented. Training samples were drawn with probabilities inversely proportional to their class frequency, forcing the model to encounter rare "Severe" cases more frequently during training.
- Ensemble Fusion: The outputs (logits) of the ten models were integrated. While majority voting was tested, the final KneeXNeT framework utilizes a Shallow Neural Network (FCN) as a meta-learner. This two-layer fully connected network learns to optimally weight the predictions of the individual models to make a final call.
- Explainability: Smooth-GradCAM++ was employed to generate heatmaps, highlighting the radiographic regions most influential to the model's classification.
Datasets
The study utilized the Osteoarthritis Initiative (OAI) dataset, a gold-standard public resource comprising 8,260 knee X-ray images from 4,796 participants. The data was split into training, validation, and test sets in a 7:1:2 ratio. As illustrated in Figure 1, the dataset exhibits a significant class imbalance, with Grade 0 and Grade 2 being far more prevalent than Grade 1 or Grade 4.
Key Results
Individual models reached a peak accuracy of 0.69 in baseline tests. The introduction of weighted sampling (Experiment 2) improved the performance of several architectures, with DenseNet-161 reaching an accuracy of 0.70 (Figure 2). The final KneeXNeT ensemble, utilizing the FCN meta-learner, achieved the highest overall accuracy of 0.72.
Beyond overall accuracy, the ensemble approach demonstrated lower variance and more consistent performance across all classes. Notably, the Smooth-GradCAM++ visualizations confirmed that the model focuses on the joint space between the femur and tibia—the primary clinical indicator used by radiologists—proving the model aligns with established medical practice.
Discussion and Conclusion
Despite the success of KneeXNeT, Class 1 (Doubtful) remains the most challenging to classify due to the minute radiographic differences between "Healthy" and "Doubtful" states. Future work may explore merging Class 1 with adjacent categories or utilizing ordinal loss functions to better reflect the progressive nature of OA.
KneeXNeT demonstrates that ensemble-based meta-learning, combined with targeted sampling strategies, can significantly improve the reliability of automated OA grading. By providing interpretable visual evidence, the framework offers a promising tool for assisting clinicians in radiographic evaluations and reducing diagnostic subjectivity.
Summary updated