Abstract
Event cameras are often touted as privacy-preserving sensors because they record asynchronous changes in light intensity rather than traditional RGB frames. However, recent advancements in "event-to-image" reconstruction have demonstrated that high-fidelity grayscale images can be recovered from these streams, exposing sensitive biometric data. This paper proposes an end-to-end learning framework that anonymizes event data to thwart reconstruction attacks while maintaining the utility of the data for person re-identification (ReId). By jointly optimizing for privacy and utility, we demonstrate that it is possible to track individuals across multiple cameras without compromising their visual anonymity.
Introduction
As surveillance systems increasingly adopt neuromorphic event cameras for their high dynamic range and low latency, the assumption of inherent privacy has been challenged. Attackers can use deep learning models to invert event streams into recognizable images, potentially identifying individuals by their faces or specific clothing textures. The goal of this research is to decouple "identification" (knowing who a person is) from "re-identification" (matching the same person across different views).
Methodology
The proposed framework utilizes a joint learning approach consisting of three primary modules designed to balance privacy and task utility.
1. Event Anonymization Block ($E_{an}$)
This module is a convolutional autoencoder that processes raw event data, represented as a 3D voxel grid. It transforms the data into an anonymized latent space that retains structural information but discards fine-grained details.
2. Image Reconstruction Block ($E_{rec}$)
Serving as a proxy for a potential attacker, this block attempts to reconstruct images from the anonymized stream using a pre-trained E2VID model. During training, the system seeks to maximize the loss of this block, effectively teaching the anonymizer to produce data that is uninterpretable by reconstruction algorithms.
3. Person Re-Id Block ($E_{reid}$)
This block extracts features from the anonymized events to perform the downstream task of person re-identification. It is trained to minimize ReId loss, ensuring that the "scrambled" data still contains enough information for consistent tracking.
Optimization Strategy
The framework is optimized using a composite loss function:
- Structural Loss ($L_{struct}$): Maintains the basic human silhouette using Structural Similarity Index (SSIM).
- Reconstruction Loss ($L_{rec}$): Minimized to ensure the proxy attacker fails to recover identifiable features.
- ReId Loss ($L_{reid}$): Minimized to preserve the accuracy of the tracking task.
Experimental Evaluation
The method was evaluated on a synthetic dataset (SoftBio) and a newly introduced real-world dataset, Event-ReId. The Event-ReId dataset is the first of its kind, featuring 33 subjects captured across four different Prophesee event camera views in a surveillance environment.
Privacy Performance
The anonymization significantly degraded the quality of reconstructed images. The SSIM dropped from 0.548 to 0.384. Crucially, face recognition performance using the state-of-the-art ArcFace model fell to an Area Under the Curve (AUC) of 0.53, which is statistically equivalent to a random guess.
Utility Retention
Despite the heavy anonymization, the system maintained high performance for its intended task. On the real-world Event-ReId dataset, Rank-1 accuracy only experienced a minor decrease of approximately 3.5% (from 63.7% to 59.2%) compared to non-anonymized data. This suggests that the features required for ReId—such as gait and body proportions—are distinct from the biometric features required for identification.
Conclusion
This work demonstrates that privacy and utility in event-based vision are not mutually exclusive. By specifically targeting and destroying features used for identification while preserving structural gait information, we provide a robust defense against inversion attacks. Future work will focus on reducing the computational overhead for edge deployment and scaling the Event-ReId dataset to include a larger number of subjects.
Summary updated