Abstract
Large language models (LLMs) like BERT have become essential tools in clinical Natural Language Processing (NLP), often achieving state-of-the-art performance when pretrained on Electronic Health Records (EHR). However, EHR data contains highly sensitive Protected Health Information (PHI). This paper explores whether BERT models pretrained on non-deidentified clinical notes inadvertently memorize and leak sensitive patient-condition pairs. Through a series of adversarial attacks—ranging from simple probing to complex generative methods—the authors find that while basic statistical baselines are hard to beat, sophisticated generative techniques reveal genuine privacy vulnerabilities.
Introduction
The integration of LLMs into clinical workflows offers significant benefits but raises critical privacy concerns. While researchers typically de-identify EHR data before sharing, the process is often incomplete or labor-intensive. A pivotal question emerges: Is it safe to share the weights of a model trained on private, non-deidentified data? Previous research has shown that generative models like GPT-2 can "memorize" training snippets. This study investigates if BERT, a masked language model widely used in medicine, similarly risks exposing patient identities and their medical conditions.
Methodology
The researchers developed a "battery of attacks" to extract patient-condition associations from a trained BERT model. These strategies include:
- Cloze Tasks: Using templates such as
"[Name] is a patient with [MASK]"to see if the model predicts the correct condition. - Probing Classifiers: Training shallow neural networks on BERT’s internal representations to determine if they encode patient-specific medical facts.
- Cosine Similarity: Measuring the vector distance between name and condition embeddings.
- Generative Attacks: Utilizing Gibbs Sampling to treat BERT as a Markov Random Field, identifying sequences uniquely memorized by the clinical model compared to a standard BERT baseline.
Datasets
The study utilized the MIMIC-III dataset, which was "re-identified" for experimental purposes by replacing de-identification markers with synthetic names from US Census data. Medical conditions were defined using both ICD-9 billing codes and MedCAT-extracted mentions.
Results and Discussion
A primary finding of the study is the strength of the Frequency Baseline. In many instances, an attack that appeared successful was actually just reflecting general medical statistics (e.g., the high prevalence of hypertension) rather than specific patient links.
As shown in the results, the Frequency Baseline often outperformed the model's specific predictions in condition recovery (A@10). This suggests that BERT primarily learns population-level statistics rather than individual "Patient X has Condition Y" mappings.
Key Insights
- Contextual Robustness: BERT’s contextualized embeddings are more resistant to simple similarity-based extraction than static word embeddings.
- Name Insertion Stress Test: Even when names were inserted into every sentence of the training data to maximize exposure, simple probes failed to reliably link names to conditions.
- Generative Vulnerability: The most significant threat came from generative attacks. In the "Name Insertion" model, these attacks successfully recovered true patient-condition pairs approximately 4.17% of the time.
Conclusion
While BERT does not appear to leak sensitive data through casual queries, it is not immune to sophisticated adversarial efforts. The potential for unintended memorization, especially as model sizes increase, necessitates caution. The researchers recommend against releasing models trained on non-deidentified sensitive data and have shared their experimental framework to encourage the development of better privacy-preserving techniques.
Summary updated