Automatic selection of radiological protocols using machine learning

Akshay U. Bhat, George Shih, Ramin Zabih2011 · DOI

TL;DR

This paper tackles the challenge of automatically selecting the correct imaging protocol for CT scans based on free-text patient histories. By leveraging medical ontologies and graph-based semi-supervised learning, they achieve a significant improvement over baseline methods, suggesting a promising avenue for improving diagnostic accuracy and reducing unnecessary repeat scans.

What Problem Does This Paper Address?

Radiological imaging, particularly CT scans, involves a complex set of parameters that must be precisely configured for each patient to ensure diagnostic quality. Currently, this crucial step relies on radiologists interpreting textual patient histories provided by referring physicians. However, the sheer volume of possible parameters and the nuances of medical conditions make this a challenging task. Incorrectly set protocols can lead to non-diagnostic images, necessitating repeat scans, which carry risks (especially with ionizing radiation) and incur significant costs. This paper aims to automate this protocol selection process, making it more efficient and potentially more accurate, especially in settings where radiologist availability is limited.

Methods

The authors propose a machine learning approach that leverages existing medical knowledge resources. The core idea is to represent medical concepts and their relationships as a graph.

  1. Ontology and Parsing: They utilize the Unified Medical Language System (UMLS) ontology, a comprehensive resource of medical concepts and their relationships, as the foundation for their graph. The National Library of Medicine's MetaMap tool is employed to parse free-form textual patient histories and map them to concepts within the UMLS ontology.
  2. Graph Construction: The UMLS ontology's concepts form the nodes of their graph. Relationships like "part_of" (for anatomy) and "has_finding_site" (linking diseases to body parts) are used to create edges between these nodes.
  3. Semi-Supervised Learning: Since obtaining labeled data (patient histories with confirmed correct protocols) is easier than fully supervised learning, they opt for a semi-supervised approach. They use the "adsorption" algorithm, a graph-based technique, to propagate protocol labels from a training set of radiologist-assigned protocols across the concept graph. This allows the model to learn associations between concepts and protocols, even for concepts not directly present in the initial training data.
  4. Protocol Prediction: For a new patient history, MetaMap identifies relevant concepts. The algorithm then aggregates the protocol "estimates" from these linked concepts to predict the most likely protocol.

Datasets

The primary dataset used in this study consists of approximately 1,000 abdominal CT studies from New York Presbyterian Hospital. Each study includes a short clinical history (10-30 words) and a protocol assigned by an attending radiologist. The authors note that about 20% of their dataset consists of unlabeled data (patient histories without assigned protocols), which is crucial for their semi-supervised learning approach. For evaluation, they used a stratified sample to create a more balanced representation of protocols, as their initial dataset was dominated by one common protocol (ONC1).

Key Insights and Ideas

  • Leveraging Existing Knowledge: The paper demonstrates the power of using established medical ontologies (UMLS) and NLP tools (MetaMap) to inject domain expertise into a machine learning model, rather than learning everything from scratch.
  • Graph-Based Reasoning: Representing medical knowledge as a graph allows for the propagation of information and learning of relationships between seemingly disparate concepts, which is crucial for understanding complex medical conditions.
  • The Value of Unlabeled Data: The success of their semi-supervised approach highlights the significant utility of readily available unlabeled medical data for training robust models in this domain.
  • Bridging Syntactic Gaps: The example of "abdominal pain with diarrhea" and "Crohn's disease, weight loss" illustrates how the model can infer connections between histories that are syntactically different but semantically related, a key challenge in NLP for medical text.

Key Results

In their experimental setup, the proposed machine learning method achieved a prediction accuracy of 60% for selecting the correct protocol. Furthermore, the radiologist's chosen protocol was within the top two predictions 75% of the time. This significantly outperforms a baseline method that simply predicts the most popular protocol, which achieved only 20% accuracy.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Simplification of Patient History: The current model primarily uses the "Medical History" field and does not incorporate other potentially valuable information like lab tests or patient demographics.
  • Radiologist Agreement: There's no guarantee that the radiologist's initial protocol selection was definitively correct. Future work includes a blinded review of these decisions by experienced radiologists.
  • Detecting Incorrect Protocols: The paper suggests investigating methods to detect incorrect protocols, such as identifying instances where similar CT exams are repeated shortly after.
  • Broader Protocol Set: While they focused on the 7 most common protocols due to data sparsity for others, expanding to a larger set of protocols is a natural next step.
  • Real-time Application: The potential for this system to assist in emergent care settings where radiologists are not immediately available is a significant motivation for further development.

Summary updated

Abstract

Medical imaging modalities, such as computed tomography (CT), have a large number of parameters that must be correctly set to produce a diagnostic image. In current clinical practice this is done with input from a radiologist, relying on the patient history provided in textual form by the referring physician. Since the set of parameters is so extensive, radiologists choose from a limited number of protocols, each of which is suited to a group of diseases. We propose a machine learning approach automate to this process, relying on the free-form textual input provided by the referring physician. We exploit an ontology built by the National Library of Medicine to provide domain expertise, as well as an associated parser that maps free-form text into this ontology. We use a graph-based semi-supervised learning technique, where the nodes of the graph are concepts from the ontology and the labels are protocols. The semi-supervised learning approach is motivated by the easy availability of unlabeled training data, for example from patient histories where the protocol is unknown, or even patient histories who did not receive imaging. In our initial experiments we used the adsorption algorithm of [2], running on a database of 1,000 patients who were assigned a protocol by radiologists at New York Presbyterian Hospital. On a stratified sample from our dataset we predicted the protocol selected by a radiologist 60% of the time, compared with a baseline accuracy of 20% achieved by always predicting the most popular protocol. Our results suggest that modern machine learning and NLP techniques show considerable promise for solving this important clinical problem.
BibTeX
@misc{bhat2011automatic,
  author = {Akshay U. Bhat and George Shih and Ramin Zabih},
  title = {{Automatic selection of radiological protocols using machine learning}},
  year = {2011},
  doi = {10.1145/2023582.2023591},
}
Figure