TL;DR
This paper introduces MedVidDeID, a novel six-stage pipeline for automatically de-identifying audio-video (AV) clinical encounter recordings. It addresses the critical need for privacy preservation in healthcare AV data, offering a significant reduction in manual effort and time compared to traditional methods, with success rates up to 97.5%.
What Problem Does This Paper Address?
The increasing use of audio-video (AV) data in healthcare for patient care, training, and research presents a significant privacy challenge. This data, often containing Protected Health Information (PHI), is difficult to de-identify using traditional methods that focus on text or images. Faces, voices, and environmental details in AV recordings can all reveal sensitive patient information. Manual de-identification is time-consuming, labor-intensive, and prone to errors, making it impractical for the vast amounts of AV data being generated. This paper tackles the problem of creating an efficient, automated, and comprehensive pipeline for de-identifying AV healthcare data to enable secure sharing and analysis.
Methods
MedVidDeID is a modular, six-stage pipeline that integrates several open-source tools and novel methods:
- Transcript Extraction: Uses WhisperX, an advanced Automatic Speech Recognition (ASR) tool, to generate accurate transcripts with sentence-level timestamps and speaker diarization.
- Transcript De-identification: Employs PHIlter, a robust Natural Language Processing (NLP) tool, to identify and redact PHI (names, dates, locations, etc.) from the transcripts using a rule-based system and customizable lexicon.
- Audio De-identification: Utilizes a custom module called AudioScrub, which replaces identified PHI segments with tones and applies voice transformation (using WORLD) to alter pitch and timbre, making speaker re-identification difficult.
- Video De-identification: Leverages YOLOv11 for object detection and pose estimation to identify individuals and their keypoints (facial landmarks and body joints). Two modes are offered:
- Precision Privacy Preservation (PPP) Mode: Applies a Gaussian blur selectively to facial keypoints, preserving environmental context.
- Greedy Privacy Preservation (GPP) Mode: Blurs the entire frame and overlays skeletal keypoints, offering stronger privacy but obscuring more environmental detail.
- Combining Audio and Video: A custom AV synchronization module, AVSync, aligns the de-identified audio and video streams to maintain frame-level accuracy.
- Validation: A crucial manual Quality Control (QC) step where human reviewers assess the de-identified AV files against mode-specific criteria. If residual PHI is found, targeted manual de-identification is performed.
The pipeline is managed by a Pipeline Configuration Manager and uses an Artifact Management System to store intermediate outputs for debugging and auditing.
Datasets
The pipeline was evaluated using 9.9 hours of simulated clinical AV recordings, comprising approximately 1.1 million video frames and 72,000 words. This dataset included diverse patient demographics and simulated typical clinical environment challenges like background noise and overlapping speech. PHI within the transcripts and audio was annotated to assess the effectiveness of the de-identification process.
Key Insights and Ideas
- Hybrid Approach is Key: The paper highlights that a multi-stage, multi-modal approach is necessary to tackle the complexities of AV de-identification, as no single tool can address all aspects.
- Trade-off Between Privacy and Utility: The introduction of PPP and GPP modes demonstrates a clear trade-off between the level of privacy protection and the preservation of contextual information, allowing researchers to choose based on their needs.
- Efficiency Gains are Substantial: Automating the de-identification process significantly reduces the time and human effort required, making large-scale AV data analysis more feasible.
- Manual QC Remains Critical but Can Be Optimized: While manual QC is essential for ensuring complete de-identification, the paper suggests future work on automating parts of this process to further reduce bottlenecks.
Key Results
- Success Rates:
- PPP Mode: Achieved a 50% success rate in the first round of de-identification.
- GPP Mode: Achieved a 97.5% success rate in the first round of de-identification.
- Time Efficiency:
- For a 15-minute video segment, MedVidDeID reduced de-identification time by 26.7% in PPP mode and 64.2% in GPP mode compared to manual methods.
- Manual de-identification took an average of 137 minutes per 15-minute video, while GPP took 49 minutes.
- PHI Leakage: Audio PHI leakage was approximately one word for every 584 words processed across both GPP and PPP modes.
- ASR Performance: WhisperX had an average Word Error Rate (WER) of 18.4% (excluding PHI).
- Video De-identification Error: The YOLOv11 + tracking algorithm resulted in a 7.3% frame error rate, requiring manual correction.
Limitations and Future Work
- ASR Limitations: The performance of the pipeline is limited by the inherent errors of the ASR tool (WhisperX), particularly with overlapping speech, medical terms, and filler words. This can lead to missed PHI in audio.
- PHIlter's Rule-Based Nature: PHIlter's reliance on regular expressions makes it less effective with unfamiliar text types, leading to a 0.17% PHI leakage rate in text.
- Voice Transformation Artifacts: Voice transformation can introduce noise and make voices sound robotic, potentially diminishing emotional cues.
- Video De-identification Challenges: Rapid movements, occlusions, and identifying non-facial features (tattoos, ID cards) remain challenges for video de-identification.
- Manual QC Bottleneck: The manual QC process is time-consuming and can be inconsistent.
Future work includes:
- Improving upstream ASR accuracy and reducing PHI omission/commission errors.
- Enhancing PHI detection and removal from text.
- Automating the adversarial approach to systematically detect residual PHI.
- Developing more sophisticated automated quality control systems to reduce manual review time.
- Exploring federated data collection settings and enhancing the pipeline for multisite collaborations.
- Encouraging community-driven challenges to foster advancements in AV de-identification.
Summary updated