Großer Hype um ChatGPT in der Medizin

Wilhelm Haverkamp, Nils Strodthoff, Jonathan Tennenbaum, Carsten W. IsraelHerzschrittmachertherapie 2023 · DOI

Abstract

ChatGPT, a chatbot based on a large language model, is currently attracting much attention. Modern machine learning (ML) architectures enable the program to answer almost any question, to summarize, translate, and even generate its own texts, all in a text-based dialogue with the user. Underlying technologies, summarized under the acronym NLP (natural language processing), go back to the 1960s. In almost all areas including medicine, ChatGPT is raising enormous hopes. It can easily pass medical exams and may be useful in patient care, diagnostic and therapeutic assistance, and medical research. The enthusiasm for this new technology shown even by medical professionals is surprising. Although the system knows much, it does not know everything; not everything it outputs is accurate either. Every output has to be carefully checked by the user for correctness, which is often not easily done since references to sources are lacking. Issues regarding data protection and ethics also arise. Today's language models are not free of bias and systematic distortion. These shortcomings have led to calls for stronger regulation of the use of ChatGPT and an increasing number of similar language models. However, this new technology represents an enormous progress in knowledge processing and dissemination. Numerous scenarios in which ChatGPT can provide assistance are conceivable, including in rhythmology. In the future, it will be crucial to render the models error-free and transparent and to clearly define the rules for their use. Responsible use requires systematic training to improve the digital competence of users, including physicians who use such programs.

Introduction

The rapid emergence of ChatGPT has sparked both immense hope and significant concern in medicine. For rhythmologists—specialists dealing with heart rhythm disorders and pacemakers—the tool offers potential efficiency gains in documentation and patient education. However, medicine requires a level of precision and evidence-based reliability that general-purpose Large Language Models (LLMs) aren't yet designed to guarantee. This paper addresses the gap between the "human-like" fluency of these models and their actual clinical utility, exploring whether ChatGPT can be trusted for diagnostic advice and how it navigates ethical and legal red flags like data privacy and bias.

Technical Foundations of AI in Medicine

To understand ChatGPT, it is necessary to contextualize LLMs within the broader field of Artificial Intelligence. The technology relies on the Transformer architecture, which utilizes "attention mechanisms" to process text in parallel rather than sequentially. The hierarchy of these technologies moves from general AI to Machine Learning, Deep Learning, and finally to Foundation Models (LLMs).

The hierarchy of AI technologies, placing Foundation Models (LLMs) as a subset of Deep Learning and Machine Learning.

To help medical professionals distinguish between these often-conflated concepts, the authors provide a structured glossary of terms:

Table defining key terms like Machine Learning, Deep Learning, NLP, and LLMs.

Training Data and Knowledge Cutoffs

ChatGPT (specifically versions 3.5 and 4) was trained on a massive scale, with approximately 80% of its data coming from the open internet and curated repositories like Wikipedia and Project Gutenberg. While the model likely ingested foundational medical texts, the exact composition of the training set remains a "black box." Crucially, the model discussed has a knowledge cutoff of September 2021, meaning it lacks awareness of the latest 2022 medical guidelines, a significant limitation for clinical application.

Clinical Utility and the "Hallucination" Trap

A primary concern is that ChatGPT generates text based on mathematical probability rather than a database of facts. This leads to the "hallucination" trap, where the model confidently invents plausible-sounding but entirely fake medical citations and DOIs.

In the niche of rhythmology, the model can be helpful for drafting discharge letters after catheter ablations or explaining complex stimulation protocols. However, its answers often start general and require "iterative prompting" to become useful. Because the model doesn't provide sources by default, verifying its output is labor-intensive, and "blind trust" in its human-like communication remains a major clinical risk.

Performance and Operational Scale

Despite its flaws, ChatGPT has demonstrated impressive academic performance, successfully passing the USMLE (US Medical Licensing Exam) and the European Exam in Core Cardiology. Qualitatively, it can optimize medical texts in seconds—a task that would take a human significantly longer. However, this efficiency comes at a high cost; operating these models is estimated to require roughly $700,000 per day and training involves over 10,000 GPUs.

Limitations and Future Outlook

Several critical barriers prevent the immediate, unregulated adoption of general LLMs in medicine:

  • Privacy: Inputting patient data into public cloud-based LLMs violates data protection laws.
  • Bias: Models inherit systematic distortions and prejudices present in their internet-based training data.
  • Certification: General LLMs are not currently certified medical products.

The future likely lies in specialized, certified medical LLMs that are smaller, more transparent, and trained on verified clinical data. We also anticipate "multi-modal" models capable of analyzing EKGs or X-ray images alongside text. Ultimately, while AI will provide significant assistance, the final diagnostic and therapeutic responsibility must remain firmly with the physician, requiring a new level of "digital competence" across the profession.

Summary updated

Abstract

ChatGPT, a chatbot based on a large language model, is currently attracting much attention. Modern machine learning (ML) architectures enable the program to answer almost any question, to summarize, translate, and even generate its own texts, all in a text-based dialogue with the user. Underlying technologies, summarized under the acronym NLP (natural language processing), go back to the 1960s. In almost all areas including medicine, ChatGPT is raising enormous hopes. It can easily pass medical exams and may be useful in patient care, diagnostic and therapeutic assistance, and medical research. The enthusiasm for this new technology shown even by medical professionals is surprising. Although the system knows much, it does not know everything; not everything it outputs is accurate either. Every output has to be carefully checked by the user for correctness, which is often not easily done since references to sources are lacking. Issues regarding data protection and ethics also arise. Today's language models are not free of bias and systematic distortion. These shortcomings have led to calls for stronger regulation of the use of ChatGPT and an increasing number of similar language models. However, this new technology represents an enormous progress in knowledge processing and dissemination. Numerous scenarios in which ChatGPT can provide assistance are conceivable, including in rhythmology. In the future, it will be crucial to render the models error-free and transparent and to clearly define the rules for their use. Responsible use requires systematic training to improve the digital competence of users, including physicians who use such programs.
BibTeX
@inproceedings{haverkamp2023groer,
  author = {Wilhelm Haverkamp and Nils Strodthoff and Jonathan Tennenbaum and Carsten W. Israel},
  title = {{Großer Hype um ChatGPT in der Medizin}},
  booktitle = {Herzschrittmachertherapie},
  year = {2023},
  doi = {10.1007/s00399-023-00960-5},
}
Figure