Erroneous Classification and Coding as a Limitation for Big Data Analyses: Causes and Impacts Illustrated by the Diagnosis of Clavicle Injuries

Robert Raché, L. Claude, Marcus Vollmer, Lyubomir Haralambiev, Denis Gümbel, Axel Ekkernkamp, Michael I. Jordan, Stefan Schulz-Drost, Mustafa BakırDiagnostics 2025 · DOI

TL;DR

This study reveals a surprisingly high rate of misclassification for clavicle fractures, particularly between medial and midshaft types, which significantly undermines the reliability of large-scale clinical data analyses. The findings highlight the critical need for improved diagnostic accuracy and coding practices to ensure the integrity of Big Data and AI applications in healthcare.

What Problem Does This Paper Address?

The increasing reliance on Big Data and Artificial Intelligence (AI) in healthcare promises revolutionary insights and improved patient care. However, the accuracy of these powerful tools is fundamentally dependent on the quality of the input data. This paper tackles a critical data quality issue: diagnostic misclassification. Specifically, it investigates how often clavicle fractures are incorrectly identified and coded, and what factors contribute to these errors. Clavicle fractures are common injuries, and their accurate classification is crucial for understanding their epidemiology, guiding treatment, and, importantly, for ensuring that large datasets used in Big Data and AI analyses are not built on faulty foundations. The authors hypothesize that misclassification is more prevalent than commonly assumed, especially between medial and midshaft fractures, and that this can lead to erroneous conclusions in research.

Methods

This was a retrospective, bicentric study analyzing patient data from two Level 1 trauma centers between 2008 and 2019. The researchers meticulously re-evaluated radiological images (X-rays, CT, MRI) for 1503 patients who had initially been diagnosed with clavicle fractures or related joint dislocations. They then compared these "ground truth" diagnoses with the original ICD-10 coded diagnoses, discharge summaries, and radiological reports. The study focused on identifying discrepancies and categorizing the sources of these errors, examining factors like fracture location, complexity, and the number and type of imaging modalities used. Statistical analyses, including Chi-square tests, were employed to assess the significance of observed differences and associations.

Datasets

The study utilized retrospective data from two Level 1 trauma centers, encompassing patient medical records from 2008 to 2019. The dataset included radiographic images (X-ray, CT, MRI), discharge summaries, radiological reports, and billing codes for 1503 patients with clavicle-associated shoulder girdle injuries. No new datasets were introduced; the focus was on re-evaluating existing clinical data.

Key Insights and Ideas

  • The "Big Data" Blind Spot: The study powerfully illustrates that even common injuries can be significantly misclassified, posing a direct threat to the validity of Big Data and AI analyses that rely on routine clinical coding.
  • Medial vs. Midshaft: A Diagnostic Minefield: The distinction between medial and midshaft clavicle fractures appears to be a particularly challenging area, leading to a disproportionately high rate of misclassification.
  • Errors Cascade: Misclassifications aren't isolated events; they often propagate across different documentation sources (billing, discharge summaries, reports), suggesting systemic issues in the diagnostic and coding workflow.
  • Complexity Breeds Error: As fracture complexity increases (e.g., multi-fragmentary, multi-region involvement), so does the likelihood of misdiagnosis.
  • Imaging Modality Paradox: Counterintuitively, more advanced imaging (CT, MRI) and a greater number of imaging modalities were associated with higher error rates, suggesting that interpretation challenges or workflow issues might outweigh the benefits of detailed imaging in some cases.

Key Results

  • High Overall Misclassification: Out of 1855 initial injury diagnoses, 1846 were confirmed upon review, indicating a small overall reduction in detected injuries but masking significant subgroup errors.
  • Medial Fracture Misclassification: A staggering 82.8% of initial diagnoses for medial clavicle fractures were incorrect. Only 5.2% of actual medial fractures were correctly identified initially.
  • Midshaft and Lateral Fracture Misclassification: Midshaft fractures had a misclassification rate of 42.5%, and lateral fractures 34.2%.
  • Error Sources: Billing codes were the most error-prone category (64.3% error rate), followed by discharge summaries (36.1%).
  • Interconnected Errors: Cases with discharge summary errors often had errors in other categories (over 75%), and billing errors co-occurred with other errors in over half the cases.
  • Imaging Modality Impact: The error rate increased with the number of imaging modalities used: 19.7% with one, 30.5% with two, and 40.7% with three.

Limitations and Future Work

The authors acknowledge several limitations:

  • Hospitalized Cases Only: The study focused on hospitalized patients, potentially excluding less complex, conservatively treated outpatient cases that might have lower misclassification rates.
  • Retrospective Design: The retrospective nature means conclusions are correlational, not causal, and the study is limited by the quality and completeness of existing documentation. It's difficult to pinpoint the exact source of error propagation.
  • Distinguishing Lateral Fractures and AC Joint Dislocations: The paper notes that differentiating between lateral clavicle fractures and acromioclavicular joint dislocations can be challenging, and the absence of a separate joint dislocation code might not always represent an error, especially in combined injuries.

Future work suggested includes:

  • Prospective Studies: To validate findings and better understand the timing and impact of errors.
  • Clinical and Economic Consequences: Exploring the broader implications of misclassification on patient outcomes, treatment pathways, and healthcare costs.
  • Broader Injury Types: Investigating if similar misclassification rates exist for other types of injuries.
  • AI Integration: Exploring how AI can be used to both identify and potentially prevent misclassifications, and how AI systems perform on datasets with pre-existing inaccuracies.
  • Harmonization: Investigating how AI can help standardize diagnostic classifications across different healthcare systems globally.

Summary updated

Abstract

<b>Background/Objectives</b>: Clavicle injuries are common and seem to be frequently subject to diagnostic misclassification. The accurate identification of clavicle fractures is essential, particularly for registry and Big Data analyses. This study aims to assess the frequency of diagnostic errors in clavicle injury classifications. <b>Methods</b>: This retrospective study analyzed patient data from two Level 1 trauma centers, covering the period from 2008 to 2019. Included were cases with ICD-coded diagnoses of medial, midshaft, and lateral clavicle fractures, as well as sternoclavicular and acromioclavicular joint dislocations. Radiological images were re-evaluated, and discharge summaries, radiological reports, and billing codes were examined for diagnostic accuracy. <b>Results</b>: A total of 1503 patients were included, accounting for 1855 initial injury diagnoses. In contrast, 1846 were detected upon review. Initially, 14.4% of cases were coded as medial clavicle fractures, whereas only 5.2% were confirmed. The misclassification rate was 82.8% for initial medial fractures (<i>p</i> < 0.001), 42.5% for midshaft fractures (<i>p</i> < 0.001), and 34.2% for lateral fractures (<i>p</i> < 0.001). Billing codes and discharge summaries were the most error-prone categories, with error rates of 64% and 36% of all misclassified cases, respectively. Over three-quarters of the cases with discharge summary errors also exhibited errors in other categories, while billing errors co-occurred with other category errors in just over half of the cases (<i>p</i> < 0.001). The likelihood of radiological diagnostic error increased with the number of imaging modalities used, from 19.7% with a single modality to 30.5% with two and 40.7% with three. <b>Conclusions</b>: Our findings indicate that diagnostic misclassification of clavicle fractures is common, particularly between medial and midshaft fractures, often resulting from errors in multiple categories. Further prospective studies are needed, as accurate classification is foundational for the reliable application of Big Data and AI-based analyses in clinical research.
BibTeX
@inproceedings{rache2025erroneous,
  author = {Robert Raché and L. Claude and Marcus Vollmer and Lyubomir Haralambiev and Denis Gümbel and Axel Ekkernkamp and Michael I. Jordan and Stefan Schulz-Drost and Mustafa Bakır},
  title = {{Erroneous Classification and Coding as a Limitation for Big Data Analyses: Causes and Impacts Illustrated by the Diagnosis of Clavicle Injuries}},
  booktitle = {Diagnostics},
  year = {2025},
  doi = {10.3390/diagnostics15020131},
}
Figure