Robust Privacy Amidst Innovation with Large Language Models Through a Critical Assessment of the Risks

Yao-Shun Chuang, Atiquer Rahman Sarkar, Noman Mohammed, Xiaoqian JiangJ. Am. Medical Informatics Assoc 2024 · DOI · OpenReview

TL;DR

This paper explores generating privacy-preserving synthetic clinical notes using large language models (LLMs) by comparing different generation methods. They found that a "keyword-based" approach offers a strong balance between data utility (for tasks like ICD-9 coding) and privacy, outperforming simpler "one-shot" generation methods that risk exposing more sensitive information.

What Problem Does This Paper Address?

The increasing digitization of healthcare through Electronic Health Records (EHRs) offers immense potential for research and improved patient care. However, these records contain highly sensitive Protected Health Information (PHI), making data sharing for research a significant privacy challenge. While Natural Language Processing (NLP) techniques can help de-identify PHI, current methods often struggle to fully comply with privacy regulations like HIPAA, leading to a scarcity of accessible, high-quality datasets. This limits research and the development of AI models in healthcare. The paper addresses this by investigating how LLMs can generate synthetic clinical notes that are both privacy-preserving and retain enough utility for downstream tasks, thus facilitating safer data sharing.

Methods

The researchers experimented with three LLMs: GPT-3.5, GPT-4, and Mistral 7B. They employed three text generation approaches:

  • One-shot: Directly using original (de-identified) clinical notes as prompts for the LLMs.
  • Normalized One-shot: Similar to one-shot, but the input text was first normalized by removing numbers and symbols, keeping only alphabetic words.
  • Keyword: This method involved extracting key phrases from normalized notes using unsupervised keyword extraction algorithms (KP-miner and YAKE), then using these keywords to generate synthetic notes. This approach aims to retain essential information while stripping away potentially identifying details.

Privacy was assessed by analyzing the occurrence and co-occurrence of PHI in the generated notes. Utility was evaluated using an ICD-9 coding classification task. Text quality was measured using ROUGE and Cosine Similarity metrics to compare synthetic notes with original ones.

Datasets

The study utilized a subset of the MIMIC III database, specifically 9,817 discharge summaries. This dataset was filtered to focus on single primary diagnoses from eight major disease categories. The researchers also used re-identified versions of this dataset to compare privacy and utility outcomes.

Key Insights and Ideas

  • Keyword generation is the privacy champion: This method significantly reduced PHI occurrence and co-occurrence compared to the more direct "one-shot" approaches.
  • "One-shot" is a privacy risk: Directly feeding original notes into LLMs, even after de-identification, led to higher rates of PHI exposure, especially for geographic location and dates.
  • Re-identified data can be more useful: Surprisingly, synthetic notes generated from re-identified data (where placeholders were filled with dummy PHI) often showed better utility (e.g., in ICD-9 coding) than those from de-identified data. This suggests a potential avenue for enhancing synthetic data utility without compromising privacy.
  • Geographic location is a persistent challenge: This PHI category was the most prevalent across models and generation methods, even in keyword-based approaches, highlighting its inherent difficulty in anonymization.
  • LLMs can sometimes "over-remember": Unique identifiers, which are typically rare in general LLM training data, were sometimes fully extracted and reappeared in synthetic notes, posing a privacy risk.

Key Results

  • Privacy: The Keyword generation method showed the lowest PHI occurrence, with geographic location being the primary remaining risk. One-shot generation had the highest PHI exposure.
  • Utility (ICD-9 Coding): The Normalized One-shot generation achieved the highest classification accuracy, with some models even surpassing the performance of real, de-identified benchmark data. Keyword generation showed slightly lower performance but offered better privacy.
  • Text Similarity: Mistral 7B in the Normalized One-shot generation achieved the highest ROUGE scores, indicating good text quality. Cosine Similarity results showed that Keyword and One-shot approaches generally maintained better semantic similarity than Normalized One-shot, especially when using re-identified data.

Limitations and Future Work

The authors acknowledge several limitations:

  • Comparison with existing NER models: Direct comparison with established Named Entity Recognition (NER) models for PHI detection was not feasible due to the novel evaluation approach used in this study.
  • Real-world scenario nuances: The study used a simplified dataset (single coding scenarios) and may not fully capture the complexity of real-world EHR data, which often involves multi-coding and nuanced language.
  • LLM variability: The performance of LLMs can vary significantly, and smaller models sometimes outperform larger ones, making standardized evaluation crucial.
  • Rapidly evolving field: The fast pace of LLM development means findings may quickly become outdated, necessitating continuous evaluation.

Future work could involve exploring alternative coding classifications, incorporating more complex multi-coding scenarios, and investigating emerging technologies for even more robust privacy and utility in synthetic EHR generation.

Summary updated

Abstract

This study examines integrating EHRs and NLP with large language models (LLMs) to improve healthcare data management and patient care. It focuses on using advanced models to create secure, HIPAA-compliant synthetic patient notes for biomedical research. The study used de-identified and re-identified MIMIC III datasets with GPT-3.5, GPT-4, and Mistral 7B to generate synthetic notes. Text generation employed templates and keyword extraction for contextually relevant notes, with one-shot generation for comparison. Privacy assessment checked PHI occurrence, while text utility was tested using an ICD-9 coding task. Text quality was evaluated with ROUGE and cosine similarity metrics to measure semantic similarity with source notes. Analysis of PHI occurrence and text utility via the ICD-9 coding task showed that the keyword-based method had low risk and good performance. One-shot generation showed the highest PHI exposure and PHI co-occurrence, especially in geographic location and date categories. The Normalized One-shot method achieved the highest classification accuracy. Privacy analysis revealed a critical balance between data utility and privacy protection, influencing future data use and sharing. Re-identified data consistently outperformed de-identified data. This study demonstrates the effectiveness of keyword-based methods in generating privacy-protecting synthetic clinical notes that retain data usability, potentially transforming clinical data-sharing practices. The superior performance of re-identified over de-identified data suggests a shift towards methods that enhance utility and privacy by using dummy PHIs to perplex privacy attacks.
BibTeX
@article{chuang2024robust,
  author = {Yao-Shun Chuang and Atiquer Rahman Sarkar and Noman Mohammed and Xiaoqian Jiang},
  title = {{Robust Privacy Amidst Innovation with Large Language Models Through a Critical Assessment of the Risks}},
  journal = {J. Am. Medical Informatics Assoc},
  year = {2024},
  doi = {10.1093/jamia/ocaf037},
}
Figure