De-identification is not always enough

Atiquer Rahman Sarkar, Yao-Shun Chuang, Noman Mohammed, Xiaoqian Jiang

TL;DR

De-identifying clinical notes doesn't fully protect against membership inference attacks (MIAs), a privacy threat where an attacker infers if a specific record was used to train a model. The authors propose a novel method to generate synthetic clinical notes using LLMs that achieve comparable utility to real notes but also exhibit similar MIA vulnerabilities, suggesting that synthetic data alone might not be a complete privacy solution.

What Problem Does This Paper Address?

Sharing sensitive clinical data for research is crucial for advancing healthcare, but privacy concerns and regulations like HIPAA and GDPR create significant barriers. De-identification, the process of removing personally identifiable information (PII), is a standard practice to mitigate these risks. However, recent research has shown that de-identification might not be sufficient to prevent sophisticated privacy attacks. Specifically, this paper focuses on the vulnerability of machine learning models trained on de-identified clinical notes to Membership Inference Attacks (MIAs). An MIA is a privacy attack where an adversary tries to determine if a specific data record was part of the training dataset of a machine learning model. If a model is trained on sensitive clinical notes, a successful MIA could reveal that a particular patient's data was used, which could be a significant privacy breach, especially for membership-sensitive models. The paper also explores synthetic data generation as an alternative to real data, aiming to preserve utility while enhancing privacy.

Methods

The authors propose a novel approach for generating synthetic clinical notes using large language models (LLMs), specifically GPT-3.5-turbo. The process involves three main steps:

  1. Keyword/Key-phrase Extraction: They extract key phrases from real clinical notes using algorithms like KP-miner and YAKE. These key phrases are intended to capture the essential information of the note.
  2. Prompt Creation: The extracted key phrases are used to construct prompts for the LLM. The order and selection of key phrases are crucial for guiding the LLM's generation.
  3. Note Generation: The LLM (GPT-3.5-turbo) uses the crafted prompt to generate synthetic clinical notes. The authors explore using varying numbers of key phrases to balance utility and privacy.

To evaluate the utility of these synthetic notes, they employ an ICD-9 code classification task. This involves training an ICD-9 code classifier on either real or synthetic notes and then evaluating its performance. For privacy evaluation, they implement Membership Inference Attacks (MIAs). They first demonstrate that MIAs are successful even when the victim model is trained on de-identified real clinical notes. Then, they adapt the MIA methodology to attack models trained on their generated synthetic notes.

Datasets

The primary dataset used in this study is the MIMIC-III dataset. This is a publicly available dataset of de-identified critical care data, including discharge summaries. The authors utilize the discharge summaries for both training their synthetic note generation models and for training and evaluating the ICD-9 code classifier. They also use a subset of MIMIC-III for their MIA experiments.

Key Insights and Ideas

  • De-identification is not enough: The most striking finding is that even after de-identification, clinical notes are still vulnerable to membership inference attacks. This challenges the common assumption that removing PII is sufficient for privacy.
  • Synthetic notes mimic real-note privacy risks: When synthetic clinical notes are generated to closely match the utility of real notes, they also exhibit similar privacy vulnerabilities to membership inference attacks. This suggests that simply replacing real data with synthetic data might not inherently solve the privacy problem.
  • Utility-privacy trade-off: The study reinforces the concept of a privacy-utility trade-off. Generating more useful synthetic notes (e.g., by using more key phrases) can sometimes come at the cost of increased privacy risk, and vice-versa.
  • Key phrases as a potential privacy lever: The authors found that reducing the number of key phrases used to generate synthetic notes did not drastically reduce utility, suggesting that this might be a promising avenue for achieving a better privacy-utility balance.

Key Results

  • MIA on De-identified Data: The membership inference attack achieved a significant advantage (0.47 with an AUC of 0.79) when the ICD-9 classifier was trained on de-identified MIMIC-III notes. This advantage decreased as the training set size and note length were reduced, but this also came with a drop in the classifier's performance.
  • Synthetic Note Utility: The synthetic clinical notes generated using GPT-3.5-turbo achieved performance levels close to real notes on the ICD-9 code classification task. For example, micro F1 scores dropped by only 1-3 points, and macro F1 scores by 1-4 points compared to models trained on real data.
  • MIA on Synthetic Data: When the MIA was mounted on models trained with synthetic notes, the attacker's advantage dropped by 2-5 points compared to attacks on models trained with real data. This indicates a slight improvement in privacy, but not a complete elimination of the risk.
  • Impact of Key Phrases: Reducing the number of key phrases used for prompt generation (e.g., from 1000 to 250) did not significantly degrade the utility of the synthetic notes, suggesting a potential for optimizing the trade-off.

Limitations and Future Work

  • Limitations:
    • The study acknowledges that the proposed synthetic note generation approach, while effective, does not fully solve the privacy dilemma. Synthetic notes still exhibit similar privacy concerns to real notes.
  • The authors note that there is no consensus on how to measure the quality of synthetic clinical notes, making direct comparisons challenging.
  • The current MIA attack on synthetic data still shows a significant attacker advantage, indicating room for improvement.
  • Future Work:
    • Further investigation into different approaches for synthetic clinical note generation to find better utility-privacy trade-offs.
  • Exploring which specific features of clinical notes are most influential in membership inference attacks.
  • Developing more robust defenses against MIAs, especially for models trained on sensitive data.
  • Investigating the impact of evolving LLM architectures and context window limitations on synthetic data generation and privacy.

Summary updated

Abstract

For sharing privacy-sensitive data, de-identification is commonly regarded as adequate for safeguarding privacy. Synthetic data is also being considered as a privacy-preserving alternative. Recent successes with numerical and tabular data generative models and the breakthroughs in large generative language models raise the question of whether synthetically generated clinical notes could be a viable alternative to real notes for research purposes. In this work, we demonstrated that (i) de-identification of real clinical notes does not protect records against a membership inference attack, (ii) proposed a novel approach to generate synthetic clinical notes using the current state-of-the-art large language models, (iii) evaluated the performance of the synthetically generated notes in a clinical domain task, and (iv) proposed a way to mount a membership inference attack where the target model is trained with synthetic data. We observed that when synthetically generated notes closely match the performance of real data, they also exhibit similar privacy concerns to the real data. Whether other approaches to synthetically generated clinical notes could offer better trade-offs and become a better alternative to sensitive real notes warrants further investigation.
BibTeX
@misc{sarkar2024de,
  author = {Atiquer Rahman Sarkar and Yao-Shun Chuang and Noman Mohammed and Xiaoqian Jiang},
  title = {{De-identification is not always enough}},
  year = {2024},
  eprint = {2402.00179},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.00179},
}
Figure