Mining Misdiagnosis Patterns from Biomedical Literature.

Cindy Li, Elizabeth Chen, Guergana Savova, Hamish Fraser, Carsten Eickhoff

TL;DR

Researchers mined over 27 million PubMed citations to map out how diseases are commonly mistaken for one another. By constructing a directed graph of these "misdiagnosis patterns," the study provides a data-driven foundation for clinical decision support systems to help physicians catch preventable errors.

What Problem Does This Paper Address?

Diagnostic errors are a silent crisis in healthcare, affecting an estimated 12 million Americans annually and contributing to nearly 60% of malpractice claims involving serious harm. While many medical AI tools focus on identifying the most likely diagnosis based on symptoms, they rarely account for the "great mimics"—diseases that look like something else.

The authors argue that if we can quantify which diseases are most frequently confused, we can build "safety nets" into diagnostic software. These systems could prompt a physician to reconsider a diagnosis by highlighting common pitfalls (e.g., "You've diagnosed X, but in the literature, X is frequently a misdiagnosis for Y").

Methods

The researchers developed a pipeline to extract misdiagnosis relationships from the PubMed 2018 annual baseline.

  1. Linguistic Filtering: They used Python to parse nearly 28 million articles, specifically targeting titles containing phrases like "misdiagnosed as" or "masquerading as."
  2. Named Entity Resolution: Using QuickUMLS, they extracted disease names and mapped them to Universal Medical Language System (UMLS) Concept Unique Identifiers (CUIs). They focused on specific semantic types: "Disease or Syndrome" and "Neoplastic Process."
  3. Strict Selection Logic: To ensure high-quality data, they only selected titles that contained exactly two medical concepts—one preceding the "misdiagnosis phrase" (the correct diagnosis) and one following it (the incorrect diagnosis).
  4. Normalization: They used the UMLS API to handle synonyms and parent-child relationships (e.g., grouping specific types of tumors under a broader "Tumor" parent CUI) to prevent the data from becoming too fragmented.
  5. Graph Construction: Finally, they used NetworkX to build a directed, frequency-weighted graph where nodes represent diseases and edges represent the direction and frequency of misdiagnosis.

Datasets

  • PubMed 2018 Annual Baseline: A massive corpus of 27,837,540 citations.
  • UMLS (Universal Medical Language System): Used for standardized medical terminology and hierarchical relationship mapping.

Key Insights and Ideas

  • The "Great Mimic": Tuberculosis (TB) emerged as the most frequently misdiagnosed disease. It was mistaken for 46 different conditions, most commonly various forms of carcinoma. This confirms long-standing clinical observations of TB's ability to imitate other pathologies.
  • Asymmetric Relationships: Misdiagnosis is rarely a two-way street. If Disease A is often mistaken for Disease B, it does not mean Disease B is often mistaken for Disease A. This one-sided directionality suggests that certain diagnostic pathways are inherently more prone to error than others.
  • The "Many-to-Many" Problem: The study found that most diseases aren't consistently mistaken for just one other thing. Instead, a single disease tends to be misdiagnosed as many different conditions, each with a relatively low individual probability. This makes the "search space" for errors much broader than previously thought.
  • Structural Bias in Literature: The "destination nodes" (the incorrect diagnoses) were often broader categories like "Tumor" or "Malignancy," suggesting that physicians often reach a "close enough" but ultimately incorrect category before the true underlying cause is found.

Key Results

  • Extraction Yield: From 5,105 relevant titles, the team extracted 2,502 unique misdiagnosis pairs.
  • Top Misdiagnosed Disease: Tuberculosis (CUI C0041296) had the highest "out-degree" (46 different incorrect diagnoses) and the highest source frequency (68 mentions).
  • Top Incorrect Diagnosis: "Tumor" (CUI C0027651) was the most common destination node, appearing as the incorrect diagnosis 73 times.
  • Low Pair Frequency: Even for the most common errors, the normalized frequency was relatively low (usually <13%). For example, TB was misdiagnosed as carcinoma in only about 10% of its misdiagnosis cases.

Limitations and Future Work

  • Title-Only Scope: The researchers only analyzed article titles, leading to a 51% data loss due to complex title structures that didn't fit their strict "two-concept" rule. Expanding to abstracts and full-text articles using advanced NLP tools like SemRep is a logical next step.
  • Prevalence Blindness: The study measures how often a misdiagnosis is reported, not necessarily how often it occurs in the general population. Future models would need to factor in the baseline prevalence of diseases to provide more accurate risk assessments.
  • Linguistic Variety: The study relied on a limited set of phrases. Including more varied indicators of diagnostic error could significantly increase the dataset's size and diversity.
  • Clinical Integration: The ultimate goal is to integrate these weighted graphs into Electronic Health Records (EHRs) to provide real-time "diagnostic alerts" to clinicians.

Summary updated

Abstract

Diagnostic errors can pose a serious threat to patient safety, leading to serious harm and even death. Efforts are being made to develop interventions that allow physicians to reassess for errors and improve diagnostic accuracy. Our study presents an exploration of misdiagnosis patterns mined from PubMed abstracts. Article titles containing certain phrases indicating misdiagnosis were selected and frequencies of these misdiagnoses calculated. We present the resulting patterns in the form of a directed graph with frequency-weighted misdiagnosis edges connecting diagnosis vertices. We find that the most commonly misdiagnosed diseases were often misdiagnosed as many different diseases, with each misdiagnosis having a relatively low frequency, rather than as a single disease with greater probability. Additionally, while a misdiagnosis relationship may generally exist, the relationship was often found to be one-sided.
BibTeX
@inproceedings{li2020mining,
  author = {Cindy Li and Elizabeth Chen and Guergana Savova and Hamish Fraser and Carsten Eickhoff},
  title = {{Mining Misdiagnosis Patterns from Biomedical Literature.}},
  booktitle = {PubMed},
  year = {2020},
}

Loading related content...

Figure