Privacy-Protection Drone Patrol System based on Face Anonymization

Harim Lee, Myeung Un Kim, Yeong-Jun Kim, Hyeonsu Lyu, Hyun Jong Yang

Introduction

As surveillance drones become increasingly prevalent in security and patrol operations, the risk of privacy infringement through "indiscriminate shooting" grows. While drones are excellent tools for accessing hazardous or hard-to-reach areas, their high-resolution cameras often capture sensitive personal information without consent.

Traditional privacy-preserving methods, such as blurring or pixelation, present significant technical challenges for autonomous systems. Specifically, these methods can "break" a drone's computer vision systems. If a drone utilizes Simultaneous Localization and Mapping (SLAM) to navigate, it relies on extracting feature points (edges, corners, textures) from its environment. Blurring a large portion of the frame—such as a person's face—removes these critical points, potentially causing the drone to lose orientation or fail to detect obstacles. This paper introduces a real-time drone patrol system that preserves privacy without sacrificing robot perception performance by replacing detected faces with photorealistic, synthetic ones.

System Architecture

The proposed system is built upon the Robot Operating System (ROS) and splits functionality between a ground station and the patrol drone itself.

Overview of the Face Anonymizing Patrol Drone System architecture.

The hardware configuration includes a customized DJI F550 drone equipped with a ZED stereo camera and an Nvidia Jetson Xavier "companion computer." By performing anonymization on the edge (the drone itself), the system ensures that raw, un-anonymized video is never transmitted over the air, closing a major security loophole.

Proposed Methodology

The core of the system is a two-stage deep learning framework designed for real-time execution on mobile hardware.

1. Face Detection and Segmentation

The system first detects faces using FaceBoxes, a lightweight detector. To handle varying resolutions, the system employs repetitive detection to ensure faces are tightly cropped. A modified CycleGAN then translates the photorealistic face into a semantic segmentation mask, identifying key components like eyes, nose, and mouth.

2. Face Synthesis and Replacement

A second network, based on GauGAN (SPADE), takes the segmentation mask and generates a completely new, photorealistic face. This synthetic face maintains the original pose and expression but belongs to a non-existent person.

The multi-stage anonymization pipeline: from face detection to photorealistic replacement.

Technical Innovations

To ensure high-quality anonymization, the authors introduced several novel modifications:

  • VGG Perceptual Loss with Margins: To prevent the network from accidentally regenerating the original identity, a margin ($\epsilon$) was added to the loss function. This forces the generated face to be a specific "distance" away from the original in feature space.
  • Component-Level Discrimination: The discriminator investigates individual facial components rather than the whole face. This allows the generator more freedom to create a diverse "new" face while strictly maintaining the structural integrity of the original expression.

Experimental Results

The researchers utilized benchmarks including CelebA-HQ, Helen, FaceScrub, and FaceForensic, alongside custom drone footage, to validate the system.

Privacy and Utility

Quantitative privacy was measured using a Siamese network to calculate the Euclidean distance between original and anonymized faces. In all tests, the distance significantly exceeded the threshold for a "different person," confirming effective anonymization.

Crucially, the system maintained navigation stability. When comparing SLAM performance on original versus anonymized footage, the feature points extracted by ORB-SLAM2 were nearly identical. This proves the drone can calculate distances and map its environment perfectly, even with synthetic faces in the frame.

Conclusion and Future Work

This framework successfully decouples identity from structure, allowing for privacy-compliant drone patrols. While the system achieves high visual quality and maintains SLAM performance, future work is needed to address computational overhead for high-speed flight and to expand anonymization to other identifiers like clothing, tattoos, or gait. Additionally, the robustness of the GAN against adversarial "reversal" attacks remains an open area for research.

Summary updated

Abstract

In this paper, we develop a privacy-preserving UAV system that does not infringe on the privacy of people in the videos taken by UAVs. Instead of blurring or masking the face parts of the videos, we want to exquisitely modify only the face parts so that the people in the modified videos still look like humans, but they become anonymous. Doing so, the semantic information of the videos can be preserved even with the anonymization. Specifically, based on the latest generative adversarial network architecture, we propose a deep learning-based face-anonymization scheme so that each modified face part looks like the face of a person who does not actually exist. The trained face-anonymizer is then mounted on the UAV system we have implemented. Through experiments, we confirm that the developed privacy-preserving UAV system anonymizes UAV’s first-person videos so that the people in the video are not recognized as anyone in the dataset used. In addition, we show that even with such anonymized videos, the perception performance required for performing UAV’s essential functions such as simultaneous localization and mapping is not degraded.
BibTeX
@article{lee2021privacy,
  author = {Harim Lee and Myeung Un Kim and Yeong-Jun Kim and Hyeonsu Lyu and Hyun Jong Yang},
  title = {{Privacy-Protection Drone Patrol System based on Face Anonymization}},
  journal = {IEEE Access},
  year = {2021},
  doi = {10.1109/access.2021.3113186},
}
Figure