Extraction of Diagnostic Reasoning Relations for Clinical Knowledge Graphs

Vimig SocratesACL 2022

TL;DR

This paper tackles the challenge of incomplete clinical knowledge graphs (KGs) by proposing novel methods to extract diagnostic reasoning relations. They leverage a clinical reasoning ontology and a physician-facing wiki, along with consumer health resources, to enrich KGs with clinically meaningful connections beyond simple hierarchical relationships. This work is crucial for improving automated diagnostic reasoning and clinical decision support systems.

What Problem Does This Paper Address?

Clinical knowledge graphs are powerful tools for applications like question answering and recommendation systems. However, their utility is often hampered by "missing facts" – specifically, missing relations between entities. While significant progress has been made in extracting general biomedical relations (e.g., gene-disease), clinical KGs often lack crucial diagnostic reasoning relations like comorbidities, signs, and symptoms. Existing resources like SNOMED-CT, while comprehensive, primarily contain hierarchical "is-a" or "has-a" relationships, failing to capture the nuanced connections that clinicians use to reason through diagnoses. This gap limits the ability of KGs to accurately represent real-world diagnostic processes.

Methods

The authors propose a two-pronged approach to extract diagnostic reasoning relations:

  • Wiki-Based Relation Extraction (RQ1): They leverage WikEM, a domain-specific point-of-care reference wiki for emergency department physicians. The system automatically extracts an open KG by treating wikilinks as entities and section headings as relations. These extracted entities and relations are then linked to SNOMED-CT and a Clinical Reasoning Ontology (CRO). They explore both traditional NLP tools like CTAKES and BERT-based models like SapBERT for entity linking.
  • Zero-Shot Relation Classification from Consumer Education Resources (RQ2): To capture relations not present in the structured wiki format, they use consumer health resources like MedlinePlus and Merck Manuals. Here, they extract entities and relations directly from free text. They fine-tune a T5-based model (SciFive) for an autoregressive entity retrieval task and then employ zero-shot relation classification methods, incorporating auxiliary information like relation descriptions and semantic types to improve accuracy.

Datasets

  • WikEM: A domain-specific point-of-care reference wiki for ED physicians. Used for wiki-based relation extraction.
  • MedlinePlus and Merck Manuals (Consumer Edition): Curated consumer health resources used for zero-shot relation extraction from free text.
  • SNOMED-CT: A comprehensive clinical terminology used as a target knowledge base for grounding extracted relations.
  • BioRel: A large, distantly-supervised RE dataset for the biomedical domain, used for initial evaluation of the zero-shot RE system.

Key Insights and Ideas

  • Beyond Hierarchies: The core insight is that clinical reasoning relies heavily on non-hierarchical relations (e.g., "is_symptom_of," "has_complication") that are largely absent in standard clinical KGs.
  • Leveraging Structured and Unstructured Data: The paper demonstrates the value of combining structured (wiki) and unstructured (consumer health texts) data sources to capture a broader spectrum of diagnostic reasoning relations.
  • Zero-Shot Evaluation Necessity: The lack of existing labeled data for these specific diagnostic relations necessitates novel evaluation strategies, moving beyond traditional supervised methods.
  • Clinical Reasoning Ontology as a Bridge: A Clinical Reasoning Ontology (CRO) is essential for defining and standardizing the target relations, providing a framework for extraction and integration.

Key Results

  • The paper outlines a methodology for extracting diagnostic reasoning relations from both wiki-based and free-text sources.
  • They propose two evaluation strategies: clinician review of low-confidence triples and a proxy task of differential diagnosis prediction.
  • The work aims to contribute new, clinically meaningful relations to existing KGs like SNOMED-CT.

Limitations and Future Work

  • Evaluation Challenges: The primary limitation is the lack of existing labeled data for diagnostic reasoning relations, making traditional evaluation difficult. The proposed clinician review and proxy task are attempts to address this.
  • Zero-Shot Performance: While zero-shot methods are explored, their performance on these specific clinical relations is a key area for further investigation.
  • Temporality: The paper acknowledges that modeling temporality (how disease states change over time) is a crucial consideration for real-world clinical reasoning that is not fully addressed in this work.
  • Broader Evaluation: The authors suggest continuing to define tasks that better model the complexities of real-world clinical reasoning, including longitudinal aspects.

Summary updated

BibTeX
@inproceedings{socrates2022extraction,
  author = {Vimig Socrates},
  title = {{Extraction of Diagnostic Reasoning Relations for Clinical Knowledge Graphs}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure