PerceptAnon: Exploring the Human Perception of Image Anonymization Beyond Pseudonymization for GDPR

Kartik Patwari, Chen-Nee Chuah, Lingjuan Lyu, Vivek Sharma

TL;DR

PerceptAnon is a novel, learning-based metric designed to evaluate image anonymization by aligning with human perception rather than relying solely on pixel-level changes. By moving beyond simple "pseudonymization" (e.g., hiding faces) to a holistic "global" assessment of an image, it provides a more robust framework for meeting strict privacy standards like the GDPR.

What Problem Does This Paper Address?

In the world of privacy regulations, there is a critical distinction between pseudonymization and anonymization. Most current techniques focus on the former: they blur a face or mask a body, then measure success using re-identification (re-ID) rates or pixel-similarity metrics like PSNR.

However, the General Data Protection Regulation (GDPR) defines true anonymity as making personal identification "unfeasible." A blurred face is useless if the person is standing in front of their unique home, holding a specific trophy, or wearing a highly recognizable uniform. Traditional metrics fail to capture these contextual "privacy leaks."

Contextual Privacy Leaks: Even with the face blurred, the subject's unique medal and uniform remain visible, demonstrating how local pseudonymization fails GDPR's strict anonymity standards.

The authors argue that because anonymity is ultimately judged by humans, we need a metric that mirrors human perceptual judgment across the entire image, not just localized regions.

Methods

The researchers developed PerceptAnon, a metric trained on human-annotated data to quantify perceived anonymity. Their approach involved several key steps:

  1. Dual Scopes: They evaluated images under two scopes: Local (only the personally identifiable information, or PII, region is visible) and Global (the entire image is visible, but the PII is sanitized). This highlights how much background context contributes to identification.
  2. Human Annotation Setups: They collected two types of human judgments:
  3. HA1: Annotators see only the anonymized image (mimicking real-world scenarios).
  4. HA2: Annotators compare the original and anonymized image side-by-side.
  5. The Model: They trained Convolutional Neural Networks (CNNs) using two different pipelines. For HA1, a standard CNN processes the anonymized image. For HA2, a Siamese network architecture compares the original-anonymized pair.
  6. Learning Strategy: They investigated whether treating anonymity as a continuous regression problem or an ordinal classification problem (e.g., a 10-class scale) worked better.

Datasets

The authors curated a comprehensive new dataset by selecting 500 images each from four major vision benchmarks: MS-COCO, PASCAL VOC, LFW, and CelebA-HQ.

Each image was processed using four anonymization techniques:

  • Masking: Overlaying a solid black block.
  • Blur/Pixelation: Standard obfuscation.
  • Inpainting: Using DeepFillV2 to "erase" the subject and fill the background.
  • Generative: Using DeepPrivacy2 to generate a synthetic, non-existent face or body.

This resulted in a dataset of 4,000 variations, which were then scored by human annotators on a scale of 1 to 10.

Key Insights and Ideas

  • Context is a Privacy Killer: One of the most significant findings is that "Global" images consistently receive lower anonymity scores than "Local" ones. Even if a face is perfectly masked, the background often contains enough information for a human to feel the subject is still identifiable.
  • Anonymity is a Spectrum: The paper demonstrates that anonymity isn't binary (identified vs. not identified). Humans perceive degrees of anonymity, and PerceptAnon captures this "spectral nature" far better than traditional metrics.
  • Classification > Regression: Interestingly, training the model as a multi-class classifier (e.g., 10 levels of anonymity) yielded higher correlations with human judgment than treating it as a simple regression task. This suggests that human perception of privacy might be more categorical or "bucketed" than purely linear.
  • Traditional Metrics are Blind to Privacy: Metrics like PSNR, SSIM, and FID showed a lack of strong alignment with human assessments. They often focus on texture and signal noise rather than the semantic meaning that leads to identification.

Key Results

PerceptAnon significantly outperformed all baseline metrics in correlating with human scores:

  • Correlation: On the "All" dataset (HA1), PerceptAnon achieved a Spearman’s rank correlation of 0.8817, compared to just 0.7675 for LPIPS and -0.8358 for SSIM (where lower is better for anonymity).
  • Robustness: The metric maintained high performance (ρ > 0.82) even in "Leave-One-Out" tests, where the model was tested on a dataset (like CelebA) it hadn't seen during training.
  • Interpretability: Using GRAD-CAM visualizations, the authors showed that PerceptAnon actually "looks" at privacy-sensitive background cues—like recognizable objects or unique locations—to determine its score.
PerceptAnon Interpretability: GRAD-CAM heatmaps reveal that the metric correctly identifies privacy-compromising background cues, such as the unique vehicle, rather than just focusing on the masked subjects.

Limitations and Future Work

While a major step forward, the authors note several areas for growth:

  • Cultural Nuance: Privacy perception varies across cultures and demographics, which the current dataset may not fully capture.
  • Data Imbalance: Human annotators tended to cluster scores toward the higher end (partial anonymity), making it harder to train models on extremely low-anonymity examples.
  • Expanding Scope: Future work could extend this to video anonymization or include more diverse "unseen" anonymization methods to further test the metric's generalizability.

Summary updated

Abstract

Current image anonymization techniques, largely focus on localized pseudonymization, typically modify identifiable features like faces or full bodies and evaluate anonymity through metrics such as detection and re-identification rates. However, this approach often overlooks information present in the entire image post-anonymization that can compromise privacy, such as specific locations, objects/items, or unique attributes. Acknowledging the pivotal role of human judgment in anonymity, our study conducts a thorough analysis of perceptual anonymization, exploring its spectral nature and its critical implications for image privacy assessment, particularly in light of regulations such as the General Data Protection Regulation (GDPR). To facilitate this, we curated a dataset specifically tailored for assessing anonymized images. We introduce a learning-based metric, PerceptAnon, which is tuned to align with the human Perception of Anonymity. PerceptAnon evaluates both original-anonymized image pairs and solely anonymized images. Trained using human annotations, our metric encompasses both anonymized subjects and their contextual backgrounds, thus providing a comprehensive evaluation of privacy vulnerabilities. We envision this work as a milestone for understanding and assessing image anonymization, and establishing a foundation for future research. The codes and dataset are available in https://github.com/SonyResearch/gdpr_perceptanon.
BibTeX
@inproceedings{patwari2024perceptanon,
  author = {Kartik Patwari and Chen-Nee Chuah and Lingjuan Lyu and Vivek Sharma},
  title = {{PerceptAnon: Exploring the Human Perception of Image Anonymization Beyond Pseudonymization for GDPR}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=4BWCecFEcQ},
}
Figure