“Knowledge is Power”: Constructing Knowledge Graph of Abdominal Organs and Using Them for Automatic Radiology Report Generation

Kaveri Kale, Pushpak Bhattacharyya, Aditya Shetty, Milind Gune, Kush Shrivastava, Rustom Lawyer, Spriha BiswasACL 2023

TL;DR

This paper tackles the slow and error-prone process of generating radiology reports by proposing an automated system that leverages knowledge graphs (KGs) of abdominal organs. By constructing detailed KGs from existing radiology reports and using them to enhance a deep learning model, the authors achieve significant improvements in automatic report generation, promising faster and more accurate diagnoses.

What Problem Does This Paper Address?

The traditional radiology workflow involves a radiologist dictating findings to a transcriptionist, who then creates a preliminary report. This report is then reviewed and corrected by the radiologist. This multi-step process is not only time-consuming but also prone to delays and errors, especially given the high patient-to-radiologist ratio in many parts of the world. The authors aim to automate this process by creating a system that can generate a "pathological description" (the clinical findings) directly from the radiologist's dictation, thereby reducing turnaround time and potential mistakes. A key challenge is incorporating the vast and complex domain knowledge of radiology into a format that a computer can effectively use.

Methods

The core of the proposed solution lies in constructing and utilizing knowledge graphs (KGs) of abdominal organs. The process involves several key steps:

  • Ontology Creation: The authors build a "Radiology Ontology" by leveraging the RadLex lexicon and defining eight logical relations (e.g., PartOf, TypeOf, ModifierOf). This ontology provides a structured framework for representing medical entities and their relationships.
  • Radiology Dictionary Creation: A comprehensive dictionary of medical terms is created, drawing from RadLex and enriching it with terms from the corpus of radiology reports.
  • Triplet Extraction: This module extracts entity-relation-entity triplets from free-text radiology reports. It employs a multi-stage process including a word and phrase level processor, a lexical semantic supersense tagger, and a dependency parser. This is a crucial step as standard open information extraction tools struggle with the nuances of radiology text.
  • Knowledge Graph Construction: Extracted triplets are used to build organ-specific KGs. This involves creating preliminary KGs and then augmenting them with the extracted triplets, ensuring that the KGs represent a rich and detailed understanding of abdominal anatomy and pathology.
  • Radiology Report Generation: A KG-enhanced deep learning model, specifically KG-BART, is fine-tuned to generate pathological descriptions from the radiologist's dictation. This model leverages the constructed KGs to provide domain knowledge. The system also includes a "span identification" module that identifies which parts of a standard report template need to be replaced by the generated pathological description.

Datasets

The primary dataset used in this work consists of approximately 10,000 anonymized free-text radiology reports of ultrasound scans, provided by a company collaborator. This corpus is used for constructing the knowledge graphs and the radiology dictionary. A "parallel corpora" of impressions and corresponding pathological descriptions, verified by two radiologists, is used for fine-tuning the KG-BART model.

Key Insights and Ideas

  • Knowledge Graphs as a Bridge: The paper highlights the power of KGs in bridging the gap between unstructured medical text and structured, machine-readable knowledge. By explicitly representing relationships between anatomical structures, findings, and properties, KGs provide essential context for NLP models.
  • Domain-Specific NLP is Crucial: Standard NLP tools often fall short when dealing with the specialized language and complex relationships found in radiology reports. The authors demonstrate the necessity of custom triplet extraction methods and domain-specific ontologies.
  • KG Augmentation for Dynamic Knowledge: The approach of dynamically updating KGs with new information from reports ensures that the knowledge base remains current and comprehensive, reflecting the evolving understanding of medical conditions.
  • Leveraging Pre-trained Models with Domain Knowledge: The KG-BART model shows that augmenting powerful pre-trained language models with structured domain knowledge (KGs) can lead to significant performance gains in specialized tasks like radiology report generation.

Key Results

The KG-enhanced deep learning model (KG-BART) demonstrated improved performance in generating pathological descriptions compared to baseline models like T5 and BART. Specifically, the KG-BART model achieved improvements of 2%, 4%, 2%, and 2% in BLEU-3, ROUGE-L, METEOR, and CIDEr scores, respectively. The system is capable of generating a report within 30 seconds of running a scan, a significant speed improvement over the manual process. An MVP has been released to beta users for testing and evaluation.

Limitations and Future Work

The authors acknowledge that while automated metrics were used for evaluation, a human evaluation metric would have been beneficial for contextualizing the results. They plan to conduct manual evaluations in the future. The current work focuses on abdominal organs, and future work could involve generalizing the approach to other organs and scanned procedures. The authors also mention that while the MVP is available, continuous feedback from radiologists is being collected to refine the tool.

Summary updated

BibTeX
@inproceedings{kale2023knowledge,
  author = {Kaveri Kale and Pushpak Bhattacharyya and Aditya Shetty and Milind Gune and Kush Shrivastava and Rustom Lawyer and Spriha Biswas},
  title = {{“Knowledge is Power”: Constructing Knowledge Graph of Abdominal Organs and Using Them for Automatic Radiology Report Generation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure