Abstract
Large Transformers pretrained over clinical notes from Electronic Health Records (EHR) have afforded substantial gains in performance on predictive clinical tasks. The cost of training such models (and the necessity of data access to do so) coupled with their utility motivates parameter sharing, i.e., the release of pretrained models such as ClinicalBERT. While most efforts have used deidentified EHR, many researchers have access to large sets of sensitive, non-deidentified EHR with which they might train a BERT model (or similar). Would it be safe to release the weights of such a model if they did? In this work, we design a battery of approaches intended to recover Personal Health Information (PHI) from a trained BERT. Specifically, we attempt to recover patient names and conditions with which they are associated. We find that simple probing methods are not able to meaningfully extract sensitive information from BERT trained over the MIMIC-III corpus of EHR. However, more sophisticated "attacks" may succeed in doing so: To facilitate such research, we make our experimental setup and baseline probing models available at https://github.com/elehman16/exposing_patient_data_release
Introduction
Pretraining large language models on domain-specific data, like Electronic Health Records (EHR), significantly boosts performance on clinical NLP tasks. Naturally, researchers want to share these pretrained models (e.g., ClinicalBERT) with the broader community to save compute and accelerate research. However, EHRs contain highly sensitive Protected Health Information (PHI). If a model is trained on non-deidentified clinical notes, does it memorize patient names and their specific medical conditions? If so, malicious actors could extract this data from the public model weights, violating privacy laws like HIPAA. This paper systematically tests how vulnerable a clinical BERT model is to adversarial data extraction, attempting to answer the practical question: Is it safe to share the weights of a model trained on sensitive data?
Methodology
Because publicly available EHRs are already deidentified, the authors simulate a sensitive dataset by artificially injecting synthetic patient names (sampled from US Census data) into the deidentified MIMIC-III dataset. The core dataset consists of English clinical notes (Physician, Nursing, Discharge Summaries) from the MIMIC-III intensive care database. Synthetic names were assigned to 27,906 patients. Patient conditions were defined using ICD-9 Codes and MedCAT annotations.
Several BERT models were trained, including a "Name Insertion" variant where a patient's name is prepended to every sentence in their notes to maximize potential memorization. To attempt to extract PHI from the trained weights, three primary "attacks" were deployed:
- Fill-in-the-Blank (Prompting): Querying the masked language model with templates like
[CLS] Mr. [Name] has [y]to see if it predicts the correct conditiony. - Probing: Training a multi-layer perceptron (MLP) on the
[CLS]token embedding of a patient's name to predict their conditions. - Text Generation: Using Gibbs sampling to force BERT (which is not naturally an autoregressive generator) to generate text, then ranking the outputs to find memorized snippets.
- Cosine Similarities: Checking if the contextualized embeddings of a patient's name are closer in vector space to their actual conditions than to unassociated conditions.
Key Results and Insights
The study found that simple attacks generally fail to beat basic statistical baselines. While fill-in-the-blank and probing attacks perform better than random chance, they struggle to outperform a naive baseline that simply guesses the most frequent conditions in the dataset.
- Fill-in-the-blank: As shown in the results table, an empirical frequency baseline achieved an AUC of 0.926 for ICD-9 codes. In contrast, the standard BERT base model only achieved an AUC of 0.614, indicating it could not effectively link specific names to conditions beyond general population statistics.
- Probing and Embeddings: Probing the
[CLS]token to predict specific patient conditions yielded A@10 (accuracy at top 10) scores near zero. Furthermore, unlike static word embeddings (e.g., Word2Vec), BERT's contextualized embeddings did not leak patient-condition associations via cosine similarity. - Name Recognition: When testing if the model could recognize whether it had seen a specific patient's name during training, the AUC was approximately 0.50, equivalent to random chance.
- Text Generation: This method yielded the most concerning results. In the "Name Insertion" model, 4.17% of generated sentences containing a patient's name also contained one of their true medical conditions. However, this may be partially due to generic names and common conditions.
Discussion and Conclusion
The results suggest that BERT models are relatively robust against simple PHI extraction attacks. Even when patient names were artificially over-represented, simple probing methods failed to reliably extract sensitive data. However, the authors highlight several critical caveats:
- Model Architecture: This study focused on BERT. Autoregressive models (like GPT-3 or T5) are known to be more prone to memorization and likely pose a higher risk.
- Generation Attacks: Generation-based attacks appear to be the most viable threat vector and require further exploration.
- Safety First: Despite the low success of simple extraction methods in this study, the authors strongly advise researchers to err on the side of caution and only release models trained on fully deidentified data.
Summary updated