Privacy-Enhancing Person Re-identification Framework - A Dual-Stage Approach

Kajal Kansal, Yongkang Wong, Mohan S. Kankanhalli

Introduction

Person Re-identification (Re-ID) is a critical computer vision task focused on matching individuals across non-overlapping camera views. While highly effective for security and tracking, standard deep learning models for Re-ID often inadvertently capture Personally Identifiable Information (PII). This data includes fine-grained details like facial features or specific clothing patterns that allow for unauthorized identity inference or even the reconstruction of original images from feature vectors.

Potential privacy loss in Person Re-ID via identity inference and decoder reconstruction.

The PEPR-Net Framework

To address these privacy concerns, the authors propose PEPR-Net, a dual-stage framework designed to balance matching utility with robust privacy protection. The framework operates in two distinct phases: training-time de-identification and deployment-time differential privacy.

Stage 1: Privacy-Preserving Model Learning

The first stage focuses on training an encoder that is discriminative for Re-ID but invariant to specific identity signatures. This is achieved through two auxiliary components integrated into the training pipeline:

  • De-identification (De-ID) Decoder: This module attempts to reconstruct a pixelated version of the input image from the learned features. By forcing the model to target an anonymized reconstruction, the encoder learns to prioritize structural information over high-frequency PII details.
  • Adversarial-Identity (Adv-ID) Module: This module employs adversarial training to penalize the encoder if its features can be used to correctly identify a specific person. This effectively "scrubs" identity-specific signatures from the feature space.
Stage 1 of PEPR-Net: Privacy-preserving model learning using adversarial modules and de-identification decoders.

Stage 2: Controllable Privacy via Differential Privacy

During the deployment phase, PEPR-Net introduces a Gaussian noise generator to the gallery features. By applying $(\epsilon, \delta)$-differential privacy, system operators can adjust a "privacy budget" ($\epsilon$). A lower budget increases noise and privacy at the cost of some matching accuracy, providing a tunable "knob" to meet different legal requirements, such as GDPR, without retraining the model.

Evaluation and Key Results

The framework was evaluated on two standard Re-ID benchmarks: Market-1501 and CUHK03-NP. The results demonstrate a superior balance between privacy and utility:

  • Privacy Reduction: On Market-1501, PEPR-Net reduced identity leakage from 72.74% to 33.55% while maintaining a competitive mAP of 57.65% (compared to a baseline of 69.22%).
  • State-of-the-Art Performance: PEPR-Net significantly outperformed existing privacy-preserving methods. On the challenging CUHK03-NP dataset, it achieved 69.40% mAP, whereas the best previous privacy method (PIS) only reached 50.4%.
  • Feature Distribution: t-SNE visualizations confirmed that PEPR-Net features are more uniformly distributed and less clustered by identity compared to standard models, indicating successful de-identification.

Conclusion

PEPR-Net offers a robust solution for privacy-preserving person Re-ID. By combining adversarial learning during training with controllable differential privacy during deployment, it provides a flexible framework that protects individual privacy without sacrificing the core utility of the Re-ID system. Future work will explore incorporating differential privacy directly into the training process to further optimize the utility-privacy trade-off.

Summary updated

Abstract

In this work, we show that deep learning-based re-identification (Re-ID) models, albeit trained only with a Re-ID objective (i.e. if two samples belong to the same identity), encode personally identifiable information (PII) in the learned features that may lead to serious privacy concerns. In cognizance of the modern privacy regulations on protecting PII, we propose a novel dual-stage person Re-ID framework that (1) suppresses the PII from the discriminative features, and (2) introduces a controllable privacy mechanism through differential privacy. The former is achieved with a self-supervised de-identification (De-ID) decoder and an adversarial-identity (Adv-ID) module, whereas the latter mechanism leverages a controllable privacy budget to generate a privacy-protected gallery with a Gaussian noise generator. Furthermore, we introduce the notion of a privacy metric to quantify the privacy leakage in Re-ID features which is not explicitly examined in prior work. We demonstrate the feasibility of our approach in achieving a better trade-off between utility and privacy through rigorous experiments on person Re-ID benchmarks.
BibTeX
@inproceedings{kansal2024privacy,
  author = {Kajal Kansal and Yongkang Wong and Mohan S. Kankanhalli},
  title = {{Privacy-Enhancing Person Re-identification Framework - A Dual-Stage Approach}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
  doi = {10.1109/wacv57701.2024.00835},
}
Figure