KnowLab_AIMed at MEDIQA-CORR 2024: Chain-of-Though (CoT) prompting strategies for medical error detection and correction

Zhaolong Wu, Abul Hasan, Jinge Wu, Yunsoo Kim, Jason Cheung, Teng Zhang, Honghan WuNAACL 2024

TL;DR

This paper explores how Chain-of-Thought (CoT) prompting, combined with Retrieval-Augmented Generation (RAG), can improve Large Language Models' (LLMs) ability to detect and correct medical errors in clinical notes. By developing two distinct CoT-based methods and an ensemble approach, the authors achieved competitive results in the MEDIQA-CORR 2024 shared task, highlighting the potential of structured reasoning for clinical NLP.

What Problem Does This Paper Address?

The increasing adoption of Electronic Health Records (EHRs) presents a rich source of data for clinical analysis. However, these records are prone to common sense medical errors, inaccuracies, and misinformation, which can hinder their utility and potentially lead to patient harm. While Large Language Models (LLMs) show promise in processing clinical text, effectively identifying and correcting these subtle errors remains a significant challenge. The MEDIQA-CORR shared task aims to spur research into automated methods for medical error detection and correction within clinical notes. This paper tackles this challenge by investigating how advanced prompting techniques, specifically Chain-of-Thought (CoT), can enhance LLM performance in this critical domain.

Methods

The authors propose three methods, all leveraging GPT-4 and In-Context Learning (ICL) augmented with Retrieval-Augmented Generation (RAG):

  • ICL-RAG-CoT (Chain-of-Thought Prompting): This method involves a manual analysis of a subset of the training data to categorize common medical errors into three types: Diagnosis, Intervention, and Management. Based on these categories, three distinct CoT prompts are crafted. For error detection and span identification (sub-tasks 1 and 2), GPT-4 is guided through a sequence of prompts, iteratively refining its reasoning until an error is identified or the note is deemed error-free. For the Natural Language Generation (NLG) task (sub-task 3), GPT-4 is prompted independently to correct the identified error. The RAG component uses e5-large-unsupervised embeddings to retrieve the k-most similar training instances (k=4) for each input.
  • ICL-RAG-Reason (Reasoning Prompting): This approach aims to address all three sub-tasks simultaneously using a single prompt. It first prompts GPT-4 to generate a brief reason for the correctness or incorrectness of clinical notes from the training set. If an error is present, the LLM is also prompted to provide the corrected sentence and explain why the original was incorrect. This method also utilizes OpenAI embeddings for RAG, sampling 4-shot training notes that are semantically similar. The sampling is repeated three times to ensure diverse reasoning paths, and results are aggregated via majority voting.
  • Ensemble Method: This approach combines the predictions from ICL-RAG-CoT and ICL-RAG-Reason. It initially accepts predictions from ICL-RAG-CoT for sub-tasks 1 and 2, and from ICL-RAG-Reason for sub-task 3. Conflicts are resolved by identifying notes where both methods predict an error but disagree on the error sentence ID. For sub-task 3, the ensemble method then prompts GPT-4 with ICL examples to generate a corrected sentence, specifying the error sentence ID identified by ICL-RAG-CoT.

Datasets

The study utilizes datasets from the MEDIQA-CORR 2024 shared task:

  • MS Training Set: Contains 2,189 clinical notes.
  • MS Validation Set: Contains 574 clinical notes.
  • UW Validation Set: Contains 160 clinical notes.
  • Test Dataset: Derived from two sources, totaling 926 clinical notes.

The paper does not introduce new datasets but uses existing ones for training and evaluation.

Key Insights and Ideas

  • CoT Enhances Clinical Error Detection: The study demonstrates that Chain-of-Thought prompting significantly improves LLM performance in detecting and identifying medical errors compared to standard prompting, especially in few-shot settings.
  • Reasoning is Crucial for Correction: The ICL-RAG-Reason method shows that explicitly generating and utilizing reasons for correctness/incorrectness is particularly beneficial for the Natural Language Generation (NLG) task of error correction.
  • Ensemble Approach Yields Robust Results: Combining the strengths of CoT and reasoning-based methods through an ensemble approach leads to the best overall performance, achieving competitive rankings in the shared task.
  • Few-Shot Imbalance Matters: The performance of the ICL-RAG-CoT method was observed to be sensitive to the number of few-shot examples, with 2-shot and 4-shot settings performing better than 3-shot and 5-shot, suggesting potential issues with class imbalance in few-shot learning for this task.

Key Results

The ensemble method achieved a ranking of 3rd for sub-tasks 1 (binary classification) and 2 (span identification), and 7th place for sub-task 3 (NLG) among all submissions. Specifically, on the Test dataset:

  • Sub-task 1 (Error Detection): The ensemble method achieved an accuracy of 69.40%.
  • Sub-task 2 (Span Identification): The ensemble method achieved an accuracy of 61.94%.
  • Sub-task 3 (Error Correction): The ensemble method achieved a BLEURT score of 0.6541.

The ICL-RAG-CoT method achieved accuracies of 66.20% and 62.36% for sub-tasks 1 and 2 on the MS Validation set, respectively. The ICL-RAG-Reason method performed better than ICL-RAG-CoT on the NLG task.

Limitations and Future Work

The authors acknowledge that their study did not rigorously evaluate the NLG task (sub-task 3) in isolation, which contributed to their overall ranking falling towards the lower end of the top 10. They suggest that future work could involve:

  • Evaluating LLM generation against knowledge bases: Further enhancement could be achieved by verifying LLM-generated corrections against clinical and/or biomedical knowledge bases.
  • Exploring open-source LLMs: The authors aim to explore their approach for other downstream tasks in the clinical domain using open-source LLMs.
  • Improving NLG performance: While the ensemble method showed improvement, there remains scope for further enhancement in NLG performance.

Summary updated

BibTeX
@inproceedings{wu2024knowlab,
  author = {Zhaolong Wu and Abul Hasan and Jinge Wu and Yunsoo Kim and Jason Cheung and Teng Zhang and Honghan Wu},
  title = {{KnowLab\_AIMed at MEDIQA-CORR 2024: Chain-of-Though (CoT) prompting strategies for medical error detection and correction}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure