Abstract
Crowdsourcing facial annotations for sensitive datasets, such as those involving medical patients or children, presents significant privacy risks. This paper proposes a "zero-information" approach to privacy-preserving annotation by using "surrogate" face images. These images are synthesized via Generative Adversarial Networks (GANs) or 3D reconstruction models to preserve task-relevant low-level attributes (e.g., Action Units) while completely decoupling the original subject's identity. Our evaluation shows that human labelers can achieve high accuracy on these synthetic faces, particularly for complex expressions like anger, offering a scalable and ethical solution for training robust facial analysis models.
Introduction
Training deep learning models for facial analysis—such as detecting student engagement or clinical pain—requires massive, human-labeled datasets. While crowdsourcing platforms like Amazon Mechanical Turk provide a standard solution, they are often unusable for sensitive data due to privacy concerns. Traditional de-identification techniques, such as blurring or pixelation, typically destroy the subtle facial features (e.g., muscle movements or Action Units) that annotators need to provide accurate labels.
The authors propose a framework where surrogate images are generated to contain zero information about the original identity while perfectly preserving the attributes necessary for the labeling task.
Methodology
The proposed pipeline decouples identity from expression through a multi-stage process. First, a low-level classifier extracts attributes such as head pose, gender, and specific facial Action Units (AUs) from a raw image. These attributes form a vector that serves as the input for a face generator.
The researchers compared two distinct generation methods:
- 3D Face Reconstruction (Extreme3D): A geometry-based approach that estimates 3D shape parameters from 2D images to render a detailed mesh.
- Conditional BEGAN-CS: An extension of the Boundary Equilibrium GAN with Constrained Space. The authors added an auxiliary predictor network to the discriminator, forcing the generator to produce images that match the input attribute vector.
Experimental Setup
The GAN was trained on approximately 18,000 images from the GENKI-1M dataset. For evaluation, 120 images from the GENKI-4K dataset were selected for "joy" and "anger" labeling tasks. Sixty Mechanical Turk workers annotated three types of images: raw, 3D-rendered, and GAN-generated.
Key Results and Insights
The study revealed several critical findings regarding the viability of synthetic surrogates:
- Accuracy Parity in Specific Tasks: For the "anger" detection task, the difference in accuracy between raw images (79.9%) and GAN surrogates (75.5%) was not statistically significant. This suggests that for certain high-level labels, synthetic faces are as effective as real ones.
- The Consensus Effect: When filtering for high-consensus images (where >90% of workers agreed), the accuracy for surrogate images reached approximately 96%.
- Emotion Sensitivity: Performance varied by emotion. While "anger" was preserved well, "joy" saw a larger drop in accuracy (89% raw vs. 71.2% GAN). This indicates that current synthesis methods may smooth over the subtle textures required to identify certain expressions.
- Scalability of GANs: Despite being trained on a much smaller dataset (18k images) compared to the 3D model (millions of images), the GAN performed comparably, highlighting its potential as a scalable solution.
Conclusion and Future Work
This "zero-information" approach provides a stronger privacy guarantee than $k$-anonymity methods because the synthetic images contain no pixels from the source data. However, challenges remain. Errors in the initial attribute extraction phase are faithfully reproduced by the generator, leading to unlabelable images. Furthermore, current models struggle with extreme head poses and artifacts like glasses or facial hair. Future research will focus on increasing image resolution and training on more diverse datasets to improve the veracity of synthetic expressions across a broader range of human emotions.
Summary updated