Abstract
The soaring number of personal mobile devices and public cameras poses a threat to fundamental human rights and ethical principles. For example, the theft of private information such as face images by malicious third parties can lead to catastrophic consequences. While most existing protection algorithms are effective, they are often irreversible. Here, we propose a practical and systematic solution to invertibly protect face information in the full-process pipeline from camera to final users. Specifically, we design a novel lightweight Flow-based Face Encryption Method (FFEM) on the local embedded system privately connected to the camera, minimizing the risk of eavesdropping during data transmission. FFEM uses a flow-based face encoder to encode each face to a Gaussian distribution and encrypts the encoded face feature by randomly rotating the Gaussian distribution using a rotation matrix as the password. While encrypted latent-variable face images are sent to users through public channels, the password is protected through secure channels. Users can select to decode an image with fake faces from the encrypted image on the public channel, while only trusted users are able to recover the original face. Furthermore, by tuning the Gaussian ball in latent space, we can control the fairness of the replaced face on attributes such as gender and race. Extensive experiments demonstrate that our solution protects privacy and enhances fairness with minimal effect on high-level downstream tasks.
Introduction
In our hyper-connected world, personal mobile devices and public surveillance cameras are ubiquitous. This creates a massive privacy risk: if a raw image is intercepted while being uploaded to the cloud for processing, sensitive data like faces are exposed. Existing solutions have major drawbacks. Simple methods like blurring or pixelation are often irreversible and ruin the image's utility. More sophisticated "invertible" de-identification methods usually require heavy computing power, meaning the raw image must still be sent to a powerful server before it gets encrypted—leaving a window of vulnerability for "eavesdropping" during transmission. Furthermore, AI models often inherit biases regarding race and gender; protecting privacy should not come at the cost of fairness.
Methodology: Flow-based Face Encryption
The core innovation of this work is the Flow-based Face Encryption Method (FFEM), which moves encryption to the "edge"—specifically, onto local embedded systems (such as an NVIDIA Jetson Nano) directly connected to the camera. This eliminates the "transmission gap" where raw data is most vulnerable.
The pipeline consists of four primary stages:
- Detection and Frontalization: The system detects a face and adjusts its angle to a frontal view using lightweight models.
- Flow-based Encoding: Using a generative flow-based model, the face is mapped into a latent space where the data follows a standard Gaussian distribution, referred to as a "Gaussian ball." Unlike standard neural networks, flow-based models are mathematically invertible.
- Rotation Encryption: To encrypt the face, the system applies a random orthogonal rotation matrix to this Gaussian ball. This matrix acts as the "password." Because a rotated Gaussian distribution remains Gaussian, the encrypted data appears as random noise or a generic "fake" face to unauthorized parties.
- Tiered Access: The system supports different user levels. Trusted users with the password can perfectly reconstruct the original face. Privacy-aimed users can decode the data without a password to see a realistic but "fake" face that maintains pose and expression but not identity. Finally, AI models can perform tasks like action recognition directly on the encrypted latent-variable images.
Fairness and Utility Enhancement
A significant finding of this research is the ability to enhance algorithmic fairness via "temperature" control in the latent space. By shrinking the size of the Gaussian ball (adjusting the parameter $\lambda$), the system generates faces that converge toward a "mean face." This effectively neutralizes attributes like race and gender in the fake images, significantly reducing algorithmic bias.
Furthermore, the system preserves the utility of the data for downstream tasks. Action recognition models, for instance, performed just as well on encrypted images as on original ones. This suggests that facial identity is not required for many high-level AI tasks, allowing for privacy without sacrificing functionality.
Experimental Results
The authors evaluated FFEM using standard benchmarks including CelebA-HQ, FFHQ, LFW, UTK-face, and HMDB51. Key findings include:
- Privacy: Against "black-box" face recognition models (e.g., FaceNet, ArcFace), encryption reduced identification accuracy from ~95% to effectively 0%.
- Fairness: In tests for facial attractiveness bias, the mean difference between genders dropped from 14.3% in original images to 7.73% in encrypted ones.
- Efficiency: The entire process takes approximately 307 milliseconds on an embedded device, consuming only 1.3 Joules, making it viable for real-world IoT deployment.
Conclusion
FFEM provides a practical, systematic solution for invertible face protection. While the current system occasionally produces unnatural backgrounds in "fake" face generation and struggles with extreme head "roll," it represents a major step toward privacy-preserving and fair computer vision. Future work will focus on improving visual blending and architectural robustness to varied head poses.
Summary updated