TL;DR
This paper introduces a novel method for anonymizing entire human bodies in "in-the-wild" images using Generative Adversarial Networks (GANs). By leveraging dense pixel-to-surface correspondences and a new adaptive modulation technique, their approach generates realistic and diverse human bodies, significantly improving data usability for computer vision tasks while preserving privacy.
What Problem Does This Paper Address?
The increasing reliance on large datasets for training computer vision models is often hindered by privacy regulations. While existing methods can effectively anonymize faces, they leave other identifying features on the human body untouched. This paper tackles the challenging problem of full-body anonymization in realistic, unconstrained images, aiming to create anonymized datasets that are both privacy-preserving and retain sufficient data utility for downstream tasks, unlike traditional methods like pixelation which degrade data quality.
Methods
The core of the proposed method lies in "Surface-Guided GANs." The process begins with a CSE-detector that establishes pixel-to-surface correspondences for each person in an image. This surface information is then used to guide a GAN in generating a new, anonymized human body.
Key novelties include:
- Variational Surface Adaptive Modulation (V-SAM): This technique projects the input latent space of the generator into a "surface-adaptive latent space." This allows the generator to disentangle factors of variation (like clothing color) from their spatial position on the body, leading to more robust and disentangled representations.
- Discriminator Surface Supervision: The discriminator is trained to predict surface embeddings, providing higher-fidelity feedback to the generator. This surface awareness significantly improves the quality of the synthesized images.
- Anonymization Pipeline: The framework treats anonymization as an image inpainting task. After detecting and masking out the human body, the GAN, guided by V-SAM and discriminator supervision, fills in the masked region with a synthesized, anonymized human.
Datasets
- COCO-Body: A derived version of the COCO dataset, containing cropped images with a single human in the center. It includes automatically annotated CSE embeddings and dilated masks for anonymization.
- DeepFashion-CSE: Images from the In-shop Clothes Retrieval Benchmark of DeepFashion, annotated with CSE embeddings. This dataset is used for static scene synthesis.
Key Insights and Ideas
- Surface Information is Crucial for Full-Body Synthesis: Unlike face anonymization, synthesizing realistic human bodies requires understanding their 3D structure, which dense pixel-to-surface correspondences provide.
- Disentangling Appearance from Pose/Position: V-SAM's ability to create a surface-adaptive latent space is key to generating diverse appearances independent of the person's position or pose in the image.
- Discriminator Supervision Matters: Training the discriminator to understand surface geometry provides a much stronger signal for generating high-quality, realistic human bodies than semantic-based supervision alone.
- Anonymization as Inpainting: Framing the problem as inpainting allows the GAN to leverage its generative capabilities to fill in the missing human regions with novel identities.
Key Results
- Improved Image Quality: Surface guidance significantly improves image quality, as evidenced by lower FID scores compared to baselines.
- Enhanced Diversity: The method generates diverse samples, outperforming traditional anonymization techniques in terms of data usability. For instance, pixelation degraded Mask R-CNN's average precision by 14.4%, while this method only caused a 2.8% degradation.
- Better Privacy Guarantees: Compared to face anonymization and pixelation, their method shows a much lower re-identification rate on the Market1501 dataset.
- Affine Invariance: V-SAM demonstrates improved invariance to affine transformations, which is beneficial for handling variations across frames in video.
Limitations and Future Work
- Detection Imperfections: The method relies on accurate human detection, and errors in detection can compromise privacy. The authors suggest face detection as a fallback.
- Potential Identity Leaks: While reduced, identity leaks through CSE embeddings are still a possibility.
- Recognizable Artifacts: Despite improvements, some synthesized images can still appear artificial to human evaluators.
- Dataset Size: The COCO-Body dataset, while large, is considered relatively small compared to datasets used for simpler tasks like face synthesis.
- Future Directions: Combining V-SAM with task-specific modeling for pose/motion transfer could address limitations in 3D affine invariance. Further extensions with adaptive augmentation or transfer learning are also suggested. The authors also acknowledge the potential for misuse of realistic human synthesis, similar to DeepFakes, and point to ongoing research in detection and watermarking.
Summary updated