Reliability of large language model knowledge across brand and generic cancer drug names

Jack Gallifant, Shan Chen, Sandeep Jain, Pedro Moreira, Ümit Topaloĝlu, Hugo J.w.l. Aerts, Jeremy L. Warner, William La Cava, Danielle S. BittermanmedRxiv 2024 · DOI

Abstract

As large language models (LLMs) are increasingly integrated into clinical workflows, ensuring their consistency across synonymous medical nomenclature is vital for patient safety. This study evaluates the "relational awareness" of three LLMs—GPT-3.5-turbo, GPT-4-turbo, and GPT-4o—regarding brand and generic oncology drug names. While the models demonstrated high proficiency in matching brand names to their generic counterparts, they exhibited significant inconsistencies in complex clinical tasks. GPT-3.5-turbo showed notable biases toward brand names in safety and effectiveness associations. Furthermore, all models performed poorly in drug-drug interaction (DDI) detection, with accuracy rates below 26%. These findings highlight a critical gap between a model's vocabulary and its clinical reasoning, necessitating more robust assessment methods for medical AI.

Introduction

In oncology, a single therapeutic agent is often referred to by both its generic name (e.g., pembrolizumab) and its brand name (e.g., Keytruda). To a human clinician, these terms are functionally identical. However, for an LLM, these names represent different tokens that may have been encountered in different contexts during pre-training. If an LLM provides different clinical recommendations based solely on which name is used, it introduces a "relational fragility" that could compromise patient care and pharmacovigilance.

This study investigates whether LLMs maintain consistent knowledge across lexical substitutions, focusing on drug name recognition, word association, sentiment analysis, and clinical reasoning tasks such as DDI detection and adverse event diagnosis.

Methods

The researchers evaluated GPT-3.5-turbo-0125, GPT-4-turbo, and GPT-4o using a variety of datasets and tasks:

  1. Drug Name Recognition: A baseline assessment using 367 generic-to-brand and 2,516 brand-to-generic pairs derived from the HemOnc ontology.
  2. Word Association: A "list preference" task where models associated drugs with antonym pairs (e.g., "safe" vs. "unsafe").
  3. Sentiment Analysis: Models generated descriptions of drugs, which were then analyzed for sentiment using a RoBERTa-based classifier.
  4. Clinical Reasoning: The models were tested on 1,000 DDI cases from PrimeKG and 2,438 immune-related adverse event (irAE) cases generated via clinical templates and verified by physicians.

Testing was conducted across multiple "temperatures" (0.0, 0.7, and 1.0) to evaluate how stochasticity influenced consistency.

Key Insights and Results

The Vocabulary-Logic Gap

The study revealed a stark contrast between a model's ability to recognize names and its ability to use them. GPT-4o achieved a high accuracy of 97.38% in matching brand names to generics. However, this "vocabulary" success did not translate to logical consistency in downstream tasks. When the same drugs were used in clinical scenarios, the models often reached different conclusions depending on whether the brand or generic name was used.

Pre-training Biases

GPT-3.5-turbo exhibited a significant bias toward brand-name drugs. It was 43% more likely to associate brand names with "effectiveness" (OR 1.43, p < 0.05) and 76% more likely to describe them as "side-effect-free" (OR 1.76, p < 0.05). This likely reflects the prevalence of pharmaceutical marketing and promotional materials in the models' training data, which often emphasize brand names in a positive light.

Clinical Reasoning Failures

The most concerning finding was the performance in safety-critical tasks:

  • DDI Detection: Accuracy for identifying contraindicated drug-drug interactions was abysmal, falling under 26% for all models. There was no significant difference in accuracy between brand and generic names, suggesting a fundamental lack of DDI knowledge rather than just a nomenclature issue.
  • irAE Diagnosis: Consistency in diagnosing adverse events varied. GPT-3.5-turbo tended to be overly sensitive, frequently ranking the likelihood of an adverse event as "likely" or "very likely," which could lead to clinical over-diagnosis.

Sentiment Paradox

Interestingly, while GPT-3.5 associated brands with effectiveness in list tasks, it expressed more positive sentiment toward generic names in chatbot-style descriptions (brand mean 0.6703 vs. generic mean 0.9482, p < 0.0001). This suggests that LLM biases are highly sensitive to prompt structure and the specific "mode" of interaction.

Discussion and Conclusion

The results demonstrate that LLMs do not possess a unified "concept" of a drug; instead, their knowledge is tied to specific tokens. Changing a drug's name can trigger different pathways of retrieved information, leading to inconsistent and potentially dangerous clinical outputs.

While newer models like GPT-4o show improvements in consistency and a reduction in brand-name favoritism, the low accuracy in DDI detection remains a significant barrier to clinical adoption. The researchers conclude that before LLMs can be safely integrated into EHRs or used for clinical decision support, they must undergo rigorous robustness testing. Developers must ensure that medical AI can maintain "relational awareness," providing stable and accurate recommendations regardless of the specific nomenclature used.

Summary updated

Abstract

Abstract Purpose To evaluate the performance and consistency of large language models (LLMs) across brand and generic oncology drug names in various clinical tasks, addressing concerns about potential fluctuations in LLM performance due to subtle phrasing differences that could impact patient care. Methods This study evaluated three LLMs (GPT-3.5-turbo-0125, GPT-4-turbo, and GPT-4o) using drug names from the HemOnc ontology. The assessment included 367 generic-to-brand and 2,516 brand-to-generic pairs, 1,000 drug-drug interaction synthetic patient cases, and 2,438 immune-related adverse event (irAE) cases. LLMs were tested on drug name recognition, word association, drug-drug interaction (DDI) detection, and irAE diagnosis using both brand and generic drug names. Results LLMs demonstrated high accuracy in matching brand and generic names (GPT-4o: 97.38% for brand, 94.71% for generic, p &lt; 0.0001). However, they showed significant inconsistencies in word association tasks. GPT-3.5-turbo-0125 exhibited biases favoring brand names for effectiveness (OR 1.43, p &lt; 0.05) and being side-effect-free (OR 1.76, p &lt; 0.05). DDI detection accuracy was poor across all models (&lt;26%), with no significant differences between brand and generic names. Sentiment analysis revealed significant differences, particularly in GPT-3.5-turbo-0125 (brand mean 0.6703, generic mean 0.9482, p &lt; 0.0001). Consistency in irAE diagnosis varied across models. Conclusions and Relevance Despite high proficiency in name-matching, LLMs exhibit inconsistencies when processing brand versus generic drug names in more complex tasks. These findings highlight the need for increased awareness, improved robustness assessment methods, and the development of more consistent systems for handling nomenclature variations in clinical applications of LLMs. Context Summary Key objective This study aimed to assess the consistency of large language models (LLMs) in handling brand and generic oncology drug names across various tasks, including drug-drug interaction detection and adverse event identification. Knowledge generated LLMs demonstrated high accuracy in matching brand and generic names but showed significant inconsistencies in more complex tasks. Notable, models exhibited significant differences in attributing brand versus generic names to positive terms and sentiment.
BibTeX
@misc{gallifant2024reliability,
  author = {Jack Gallifant and Shan Chen and Sandeep Jain and Pedro Moreira and Ümit Topaloĝlu and Hugo J.W.L. Aerts and Jeremy L. Warner and William La Cava and Danielle S. Bitterman},
  title = {{Reliability of large language model knowledge across brand and generic cancer drug names}},
  year = {2024},
  doi = {10.1101/2024.11.08.24316949},
  howpublished = {medRxiv},
}
Figure