Verifying Machine Unlearning with Explainable AI

Àlex Pujol Vidal, Anders Skaarup Johansen, Mohammad N. S. Jahromi, Sergio Escalera, Kamal Nasrollahi, Thomas B. Moeslund

Abstract

Machine Unlearning (MU) has emerged as a critical technique for complying with privacy regulations like the GDPR by removing the influence of specific data from trained models without full retraining. However, verifying that a model has truly "forgotten" sensitive features remains a challenge. This paper proposes a novel framework that leverages Explainable AI (XAI) to visually and quantitatively verify the unlearning process. By analyzing attribution heatmaps, the researchers demonstrate that internal representations of sensitive data can be confirmed as removed, bridging the trust gap in privacy-preserving AI.

Introduction

The "right to be forgotten" under GDPR mandates that individuals can request the deletion of their personal data from AI systems. For large-scale models, retraining from scratch is often computationally prohibitive. Machine Unlearning (MU) offers an alternative by selectively removing data influence. Yet, traditional performance metrics (e.g., accuracy or error rates) are insufficient for verification; a model might fail to predict a class while still retaining its internal features. This paper addresses this "trust gap" by introducing XAI-based verification.

The Verification Framework

The authors treat unlearning as a verification problem. The proposed pipeline involves training an original model, identifying unlearning targets (sensitive features), applying an unlearning algorithm, and finally verifying the result using XAI.

The proposed framework for verifying machine unlearning using XAI metrics and human-in-the-loop satisfaction checks.

The framework utilizes several unlearning techniques, including Fine-tuning, Prune and Reinit, and "Confuse" (adding Gaussian noise to weights). Verification is performed using SIDU (Similarity Difference and Uniqueness), a gradient-free XAI method that generates saliency heatmaps. These heatmaps reveal the model's focus, allowing for the calculation of new metrics: Heatmap Coverage (HC), which measures overlap with retained vs. forgotten objects, and Attention Shift (AS), which quantifies the change in focus.

Experimental Setup

The study focuses on a regression task: counting objects in thermal images for harbor monitoring using the Long-Term Thermal-Imaging Drift (LTD) dataset. This dataset contains over 1 million images with classes including Human, Bicycle, Motorcycle, and Vehicle. The goal is to unlearn the "human" class to protect privacy while maintaining performance on other categories. This "in-the-wild" dataset presents challenges like low resolution and overlapping objects, making it a robust benchmark for MU.

Results and Visual Verification

The results indicate that MU models can achieve performance comparable to retraining from scratch while being significantly more efficient, requiring approximately 3x less training time. Crucially, XAI heatmaps provide the necessary "proof of forgetting."

Visual comparison of the unlearning process: from original data to a privacy-violating trained model, and finally to a GDPR-compliant unlearned model where focus has shifted away from human figures.

The "Confuse" method proved particularly effective, successfully dispersing the model's attention away from human-related pixels. Quantitative analysis via the Heatmap Coverage (HC) metric confirmed that unlearned models often "refocus" their attention on retained classes more effectively than models retrained from scratch. This suggests that MU can help a model reorganize its existing knowledge more efficiently than starting over.

Conclusion and Future Work

This research demonstrates that XAI is a powerful tool for auditing Machine Unlearning. By moving beyond simple performance metrics to visual attribution, developers can provide verifiable evidence of GDPR compliance. While the study currently focuses on regression tasks in surveillance, future work aims to integrate these XAI metrics directly into the training loss function to create models that are "privacy-aware" by design, actively ignoring sensitive features during the unlearning process.

Summary updated

Abstract

We investigate the effectiveness of Explainable AI (XAI) in verifying Machine Unlearning (MU) within the context of harbor front monitoring, focusing on data privacy and regulatory compliance. With the increasing need to adhere to privacy legislation such as the General Data Protection Regulation (GDPR), traditional methods of retraining ML models for data deletions prove impractical due to their complexity and resource demands. MU offers a solution by enabling models to selectively forget specific learned patterns without full retraining. We explore various removal techniques, including data relabeling, and model perturbation. Then, we leverage attribution-based XAI to discuss the effects of unlearning on model performance. Our proof-of-concept introduces feature importance as an innovative verification step for MU, expanding beyond traditional metrics and demonstrating techniques' ability to reduce reliance on undesired patterns. Additionally, we propose two novel XAI-based metrics, Heatmap Coverage (HC) and Attention Shift (AS), to evaluate the effectiveness of these methods. This approach not only highlights how XAI can complement MU by providing effective verification, but also sets the stage for future research to enhance their joint integration.
BibTeX
@inproceedings{vidal2024verifying,
  author = {Àlex Pujol Vidal and Anders Skaarup Johansen and Mohammad N. S. Jahromi and Sergio Escalera and Kamal Nasrollahi and Thomas B. Moeslund},
  title = {{Verifying Machine Unlearning with Explainable AI}},
  booktitle = {Workshops @ ICPR},
  year = {2024},
  doi = {10.1007/978-3-031-88223-4\_32},
}
Figure