Machine learning in knee arthroplasty: specific data are key—a systematic review

Florian Hinterwimmer, Igor Lazic, Christian Suren, Michael T. Hirschmann, Florian Pohlig, Daniel Rueckert, Rainer Burgkart, Rüdiger von Eisenhart‐RotheKnee 2022 · DOI

TL;DR

Machine learning (ML) is moving from medical imaging into the complex world of orthopedic outcomes, but its success depends more on data quality than quantity. This systematic review of 19 studies finds that while ML can effectively predict specific complications and costs in knee arthroplasty, the field currently struggles with complex outcomes like revision surgery due to a lack of high-quality, prospective, and domain-specific datasets.

What Problem Does This Paper Address?

While AI has revolutionized fields like radiology through automated image analysis, its application in orthopedic surgery—specifically knee arthroplasty (knee replacement)—is still in its "preliminary phase." Surgeons have access to massive amounts of data from national registries, intraoperative sensors, and robotic systems, yet we don't fully understand which ML models work best or what specific data "ingredients" are necessary to make these models clinically useful. The authors conducted this review to determine the current feasibility of ML predictions in the field and to identify the prerequisites for effective implementation.

Methods

The researchers performed a PRISMA-compliant systematic review, searching PubMed, Medline, and the Cochrane Library for studies published between 1990 and 2021. Out of 225 initial results, 19 studies met the inclusion criteria (specifically focusing on supervised ML using tabular data for outcome prediction).

To assess study quality, they used a modified Coleman Methodology Score (mCMS). Originally designed for clinical trials, the authors adapted it to evaluate ML-specific rigor, including:

  • Data screening and processing details.
  • The use of multiple performance metrics.
  • Hyperparameter fine-tuning.
  • Both mathematical and medical interpretation of the results.

Datasets

The review highlights a significant reliance on local and administrative data over large-scale clinical registries:

  • Local Databases: 11 of 19 studies used data exclusively from their own local or regional healthcare systems.
  • Administrative Databases: Included the National Inpatient Sample (NIS) and the New York Statewide Planning and Research Cooperative System (SPARCS).
  • National Registries: Only two studies utilized established medical registries (the Danish Knee Arthroplasty Registry and the Korean Society of Nephrology registry).
  • Experimental/Mobile Data: One study used data from a mobile app, and another used a tiny but high-density experimental dataset of just 6 patients.

Key Insights and Ideas

  • Specific Data > Big Data: One of the most striking findings is that "arthroplasty-specific" data (like intraoperative load readings or specific biomarkers) is often more predictive than massive, generic administrative datasets. A model trained on just 6 patients with high-density biomechanical data yielded promising results, suggesting that "data width" (relevance) can sometimes trump "data depth" (volume).
  • The Collaboration Mandate: The authors argue that the "key" to success is the close collaboration between orthopedic surgeons and data scientists. Models built without clinical context often select irrelevant parameters, while those built without technical expertise lack the fine-tuning necessary to avoid overfitting.
  • The "Revision" Difficulty: ML struggles with "general" predictions. Predicting whether a knee replacement will need a revision is hard because "revision" is a catch-all for many different failures (infection, loosening, instability).
  • Underutilized Registries: National registries contain decades of data but are rarely used for ML. The authors suggest this is because these registries weren't designed with AI in mind; their "post hoc" data architecture makes them difficult to clean and process for algorithmic training.

Key Results

The studies demonstrated "fair to good" results overall, with a median Area Under the Curve (AUC) of 0.76 (ranging from 0.57 to 0.98).

  • Complications: ML was highly effective at predicting specific, well-defined issues like end-stage renal disease (AUC 0.89) and the need for blood transfusions (AUC 0.88).
  • Costs & Logistics: Models predicting inpatient costs and length of stay (LOS) achieved a mean AUC of 0.79.
  • Functional Outcomes: Predicting patient-reported outcomes (like KOOS scores) or walking limitations yielded AUCs between 0.70 and 0.76.
  • Revision Surgery: This remained the weakest area, with AUCs as low as 0.56–0.60.
  • Methodology Quality: The median mCMS was 65/100, indicating "fair" quality. Only 6 of 19 studies were rated as "good."

Limitations and Future Work

The primary limitation is the retrospective nature of the included studies. Most models were built using data that had already been collected for other purposes, requiring extensive and often "futile" manual cleaning. Additionally, tabular clinical data has lower information density than image data, requiring much larger patient volumes to achieve high accuracy for complex outcomes.

Future Directions: 1. Prospective AI-Ready Databases: Moving away from "post hoc" analysis toward databases designed from day one for ML. 2. Longitudinal Data: Incorporating parameters recorded over time rather than single snapshots. 3. Standardized Reporting: Improving the transparency of ML methodology (fine-tuning, data screening) in orthopedic literature.

Summary updated

Abstract

III.
BibTeX
@inproceedings{hinterwimmer2022machine,
  author = {Florian Hinterwimmer and Igor Lazic and Christian Suren and Michael T. Hirschmann and Florian Pohlig and Daniel Rueckert and Rainer Burgkart and Rüdiger von Eisenhart‐Rothe},
  title = {{Machine learning in knee arthroplasty: specific data are key—a systematic review}},
  booktitle = {Knee},
  year = {2022},
  doi = {10.1007/s00167-021-06848-6},
}
Figure