Visible-Infrared Person Re-Identification (VI-ReID) is a critical task for 24/7 surveillance, matching individuals across daytime (RGB) and nighttime (infrared) cameras. However, traditional centralized training methods—which require pooling sensitive data from various cameras into a single server—pose significant privacy and security risks. This paper introduces L2RW+, a comprehensive benchmark that shifts VI-ReID toward decentralized, privacy-preserving training using Federated Learning.
The Privacy Challenge in VI-ReID
Standard VI-ReID models rely on centralized data collection, which is often a violation of privacy regulations and security protocols. Moving to a decentralized setting, where data remains on local devices, introduces three major hurdles that break existing models:
- Modality Incompleteness: A single camera may capture only RGB or only IR data, whereas standard models require paired data to learn cross-modality features.
- Identity Missing: Specific individuals are unlikely to appear in every camera's local dataset, leading to fragmented identity information.
- Domain Shift: Variations in lighting, angles, and backgrounds across different cameras create inconsistent data distributions that are difficult to align without central access.
The L2RW+ Benchmark and DPPT Methodology
To bridge the gap between idealized lab settings and real-world deployments, the authors propose the L2RW+ benchmark. This benchmark defines three privacy protocols: Camera Independence (CI) (the strictest, with no data shared between cameras), Entity Independence (EI), and Entity Sharing (ES).
To enable training under these constraints, the authors introduce Decentralized Privacy-Preserved Training (DPPT) for both image (I-DPPT) and video (V-DPPT) data. The core of this approach is the Memory Rectification Bank (MRB). Each local client maintains a memory bank of feature centers for each identity, filtering out noisy images by averaging only the Top-K best features. These local centers are uploaded to a server, aggregated into a global memory bank, and sent back to guide local training. This process aligns disparate camera representations toward a shared identity space without ever sharing raw images.
Key Insights and Results
The study reveals several advantages of the decentralized approach beyond privacy:
- Decentralization as a Regularizer: Because the model is forced to learn from isolated, diverse clients, it develops more generalized features. This makes the decentralized model more robust to adversarial attacks (like FGSM) and environmental noise (simulated rain or fog) than centralized state-of-the-art models.
- Temporal Consistency: The performance gap between centralized and decentralized training is significantly smaller for video data. The temporal consistency in video tracklets provides strong identity cues that help overcome modality and domain shifts.
- Quantifiable Privacy: Gradient inversion attacks—where an attacker tries to reconstruct training images from weight updates—showed that images from the decentralized model were significantly more obscured and lacked identifiable attributes compared to centralized models.
Performance Highlights
- Overcoming Federated Learning Baselines: Under the strict Camera Independence (CI) protocol on the SYSU-MM01 dataset, standard FedAvg achieved only 27.51% Rank-1 accuracy. The proposed I-DPPT reached 51.27%.
- Video Robustness: On the HITSZ-VCM video dataset, V-DPPT achieved 58.33% Rank-1 accuracy, significantly narrowing the gap with centralized models.
Conclusion and Future Directions
While DPPT significantly advances privacy-preserved VI-ReID, a performance gap remains relative to centralized state-of-the-art methods under the strictest privacy settings. Future work will focus on scaling decentralized training with larger datasets and developing unsupervised methods to eliminate the need for manual labeling in distributed surveillance networks.
Summary updated