TL;DR
This paper introduces MedExQA, a novel benchmark for evaluating medical Large Language Models (LLMs) on their ability to generate explanations for answers. By including multiple explanations and focusing on underrepresented medical specialties, MedExQA offers a more comprehensive assessment than existing benchmarks, revealing that current LLMs struggle with nuanced medical reasoning, especially in areas like speech-language pathology.
What Problem Does This Paper Address?
The rapid advancement of LLMs has led to their increasing application in healthcare, promising to improve patient care and streamline clinical decision-making. However, evaluating these models' understanding of complex medical knowledge is challenging. Existing medical question-answering (QA) benchmarks primarily rely on multiple-choice questions (MCQs) and classification accuracy, which fail to capture the crucial aspect of why a model arrives at a particular answer. This lack of explainability is a significant concern, as errors or "hallucinations" in medical contexts can have serious consequences. Furthermore, current benchmarks often lack coverage in specialized medical fields, such as speech-language pathology, leaving a gap in our understanding of LLM capabilities in these areas.
Methods
The authors propose MedExQA, a new benchmark designed to address these limitations. MedExQA includes two sets of explanations for each question-answer pair, aiming to provide a more robust evaluation of an LLM's reasoning. To diversify the knowledge coverage, the benchmark focuses on five underrepresented medical specialties: biomedical engineering, clinical laboratory science, clinical psychology, occupational therapy, and speech language pathology.
For evaluation, the paper employs a multi-faceted approach:
- Classification Accuracy (Logits): Standard evaluation of MCQ performance.
- Classification Accuracy (Chat): Assesses performance using string-matching on conversational outputs.
- Explanation Generation Metrics: Utilizes standard metrics like BLEU, ROUGE, METEOR, and BERTScore to evaluate the quality of generated explanations against reference explanations.
- Human Evaluation: A small-scale human evaluation was conducted by MSc-level annotators to assess the quality of explanations based on correctness and coherence.
Additionally, the paper introduces MedPhi-2, a small language model (SLM) based on Phi-2, which was further trained on medical corpora and instruction-tuning datasets. This model was developed to diversify the pool of open-source medical LLMs, which are predominantly based on Llama2.
Datasets
The primary contribution is the MedExQA dataset, which comprises 965 questions across five medical specialties. Each question-answer pair is accompanied by two distinct explanations. The dataset was manually collected from online sources and underwent rigorous preprocessing, including duplicate removal and similarity filtering.
Existing benchmarks discussed and used for comparison include:
- MMLU (Massive Multitask Language Understanding): Includes healthcare-related subjects but lacks explanations.
- MedQA: Based on medical license exams, but also lacks explanations.
- MedMCQA: Includes explanations, but nearly half the dataset lacks them, and they are not primarily used for evaluation.
Key Insights and Ideas
- Explainability is Crucial: The paper strongly advocates for evaluating LLMs beyond simple accuracy, emphasizing that understanding the rationale behind an answer is paramount, especially in high-stakes domains like medicine.
- Multiple Explanations Offer Deeper Insight: Using multiple, semantically aligned but lexically distinct explanations provides a more comprehensive assessment of an LLM's generative capabilities and aligns better with human judgment.
- Speech-Language Pathology is a Weak Spot: All tested LLMs, including advanced models like GPT-4, struggled significantly with questions from the Speech-Language Pathology domain, suggesting a need for more specialized training data and evaluation.
- Targeted Pretraining Matters: The development of MedPhi-2 demonstrates that fine-tuning smaller models with domain-specific instructions can lead to competitive performance, even outperforming larger, general-domain models in explanation generation.
Key Results
- MedExQA Performance: The MedExQA benchmark revealed that while larger models generally performed better in classification accuracy, smaller, domain-specifically trained models like MedPhi-2 showed strong performance in explanation generation.
- MedPhi-2's Prowess: MedPhi-2 outperformed medical LLMs based on Llama2-70B in generating explanations, highlighting the effectiveness of targeted pretraining and fine-tuning.
- Speech-Language Pathology Deficit: All models showed significantly lower performance in Speech-Language Pathology compared to other specialties, with SOLAR achieving the best (but still low) accuracy of 33.1%.
- Human Evaluation Alignment: Human evaluation results showed that MedPhi-2 was the best-performing open-source model and was on par with GPT3.5_1106 and GPT4_1106 in certain specialties, reinforcing the value of the benchmark.
- LLM Limitations: Even state-of-the-art models like GPT-4 exhibited limitations, particularly in the Speech-Language Pathology domain.
Limitations and Future Work
- Limited Scope: The current MedExQA benchmark focuses solely on QA tasks, limiting its applicability to real-world clinical scenarios. Future work aims to extend the benchmark to include tasks like summarizing clinical notes with explanations.
- Small-Scale Human Evaluation: The human evaluation was conducted on a small sample size (5 samples per specialty). Future work will involve increasing the number of samples and annotators for more robust assessment.
- Data Proprietary Nature: The authors acknowledge that the proprietary nature of pretraining corpora makes it challenging to definitively confirm hypotheses about why certain specialties are difficult for LLMs.
- Ethical Considerations: While the work is for research purposes, the authors emphasize that MedPhi-2 is not rigorously tested for clinical decision-making and should not be used in such contexts to mitigate risks from hallucinations.
Summary updated